question about using online comments
Hi I've been doing research using online comments on news websites. I was wondering are there any tools to extract the comments outside of copying and pasting? Chris Wiley
You can use some softwares like R or SAS or Python to request web pages as data or text files. This may have varying degrees of success. You can put these requests in loops to make multiple requests across says dates of news reporting. Then you have to clean up the data and isolate the comments from inside the text files. On Wed, May 3, 2023 at 4:49 PM Wiley, Chris via Air-L < air-l@listserv.aoir.org> wrote:
Hi I've been doing research using online comments on news websites. I was wondering are there any tools to extract the comments outside of copying and pasting?
Chris Wiley _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
A lot of news websites offer an API that makes extracting comments easiest. On Thu, May 4, 2023 at 2:23 AM Peter Timusk via Air-L < air-l@listserv.aoir.org> wrote:
You can use some softwares like R or SAS or Python to request web pages as data or text files. This may have varying degrees of success. You can put these requests in loops to make multiple requests across says dates of news reporting. Then you have to clean up the data and isolate the comments from inside the text files.
On Wed, May 3, 2023 at 4:49 PM Wiley, Chris via Air-L < air-l@listserv.aoir.org> wrote:
Hi I've been doing research using online comments on news websites. I was wondering are there any tools to extract the comments outside of copying and pasting?
Chris Wiley _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
Hi, This tutorial video might be useful [1]https://www.youtube.com/watch?v=v8Yh_4oE-Fs [2]Web Scrape Text from ANY Website - Web Scraping in R (Part 1) Web Scraping in R is super easy and useful, and in this video I scrape movies from IMDb into a data frame in R using the rvest library and then export the data frame as a csv, all in a few lines of code. This method works across many sites -- typically those that show static content -- such as Yelp, Amazon, Wikipedia, Google, and more. Part 2 ... www.youtube.com Best, Deena ____________________________ Deena Abul-Fottouh, PhD. Postdoctoral Research Fellow Digital Society Lab Political Science Department Phone icon t phone: [3](905) 525-9140 Envelope icon email: [4]abulfodm@mcmaster.ca McMaster University 1280 Main Street West Hamilton, ON L8S 4M4 McMaster University Brighter World logo __________________________________________________________________ From: Air-L <air-l-bounces@listserv.aoir.org> on behalf of Avner Kantor via Air-L <air-l@listserv.aoir.org> Sent: May 4, 2023 5:20 AM To: Peter Timusk <peterotimusk@gmail.com> Cc: air-l@listserv.aoir.org <air-l@listserv.aoir.org> Subject: Re: [Air-L] question about using online comments A lot of news websites offer an API that makes extracting comments easiest. On Thu, May 4, 2023 at 2:23 AM Peter Timusk via Air-L < air-l@listserv.aoir.org> wrote:
You can use some softwares like R or SAS or Python to request web pages as data or text files. This may have varying degrees of success. You can put these requests in loops to make multiple requests across says dates of news reporting. Then you have to clean up the data and isolate the comments from inside the text files.
On Wed, May 3, 2023 at 4:49 PM Wiley, Chris via Air-L < air-l@listserv.aoir.org> wrote:
Hi I've been doing research using online comments on news websites. I was wondering are there any tools to extract the comments outside of copying and pasting?
Chris Wiley _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers [5]http://aoir.org Subscribe, change options or unsubscribe at: [6]http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: [7]http://www.aoir.org/
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers [8]http://aoir.org Subscribe, change options or unsubscribe at: [9]http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: [10]http://www.aoir.org/
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers [11]http://aoir.org Subscribe, change options or unsubscribe at: [12]http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org Join the Association of Internet Researchers: [13]http://www.aoir.org/ References Visible links: 1. https://www.youtube.com/watch?v=v8Yh_4oE-Fs 2. https://www.youtube.com/watch?v=v8Yh_4oE-Fs 3. tel:+19055259140 4. mailto:abulfodm@mcmaster.ca 5. http://aoir.org/ 6. http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org 7. http://www.aoir.org/ 8. http://aoir.org/ 9. http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org 10. http://www.aoir.org/ 11. http://aoir.org/ 12. http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org 13. http://www.aoir.org/ Hidden links: 15. https://www.youtube.com/watch?v=v8Yh_4oE-Fs
participants (4)
-
Avner Kantor -
Deena Abul-Fottouh -
Peter Timusk -
Wiley, Chris