Downloading online forum postings and analyzing occurence of keywords in the data
Dear list-members, I'm a Utrecht University based (Netherlands) researcher working on the European http://www.mignetproject.eu/ project working on transnational digital networks, migration and gender. More specifically I'm studying knowledge and education processes across formal (university) and informal (forums, wikipedia) knowledge domains. I was wondering if anyone on the list has experiences with scraping online forum data and searching it for the occurence of keywords. Especially I'm wondering which software best suits my preferences, in selecting for instance a certain timeline to scrape as I would like to compare the curriculum of a year of university courses and their attention for diversity issues and a year of forum postings and their attention for diversity issues. Scraping software that I have used so far are: Wget Software that I have been looking at for searching keywords in digital data are: Provalis Research Wordstat NVivo Digitalmethods.net > issuecrawler I'm looking forward to any suggestions, and thank you for your time. Kind regards, Koen. Koen Leurs | Aio / PhD student Gender Studies | Research Institute for History and Culture | | Utrecht University, the Netherlands | Muntstraat 2a | 3512 EV Utrecht | Room 1.12 | www.koenleurs.net | www.uu.nl/wiredup | www.mignetproject.eu/ |
Hi Koen, I don't have any experience with scraping software, but I recently did a keyword analysis as part of a study of ethno-nationalist interaction on a Transylvanian forum. I used Wordsmith 5<http://www.lexically.net/wordsmith/version5/index.html>. It had a bit of a steep learning curve, but it was recommended by a corpus linguist and was quite efficient once I understood how it worked. I have a short paper partly based on using Wordsmith from a corpus linguistics (methodological) perspective and its relevance for microsociology (i.e., symbolic interactionism) in case you're interested. Best, patrick. On Wed, Jun 29, 2011 at 3:12 AM, Leurs, K.H.A. (Koen) <K.H.A.Leurs@uu.nl>wrote:
Dear list-members,
I'm a Utrecht University based (Netherlands) researcher working on the European http://www.mignetproject.eu/ project working on transnational digital networks, migration and gender. More specifically I'm studying knowledge and education processes across formal (university) and informal (forums, wikipedia) knowledge domains.
I was wondering if anyone on the list has experiences with scraping online forum data and searching it for the occurence of keywords. Especially I'm wondering which software best suits my preferences, in selecting for instance a certain timeline to scrape as I would like to compare the curriculum of a year of university courses and their attention for diversity issues and a year of forum postings and their attention for diversity issues.
Scraping software that I have used so far are: Wget
Software that I have been looking at for searching keywords in digital data are: Provalis Research Wordstat NVivo Digitalmethods.net > issuecrawler
I'm looking forward to any suggestions, and thank you for your time.
Kind regards,
Koen.
Koen Leurs | Aio / PhD student Gender Studies | Research Institute for History and Culture | | Utrecht University, the Netherlands | Muntstraat 2a | 3512 EV Utrecht | Room 1.12 | www.koenleurs.net | www.uu.nl/wiredup | www.mignetproject.eu/ |
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-- Patrick Williams, Ph.D. http://www.jpatrickwilliams.net
participants (2)
-
Leurs, K.H.A. (Koen) -
Patrick Williams