Quantifying Political Alignment
Our research group (made up of physicists and computer scientists) needs to evaluate the political content of short pieces of text. To this end we've constructed a rubric for classification, ranging from strong left, lean left, and neutral, to lean right, strong right, and unclassifiable. While the criteria of the rubric are rather clear cut (a 'strong' classified sentence would contain attacks on a person or group's character, hyperbolic language, or intense / fanatical expressions of support for an issue), we all suspect that this is a solved problem. I've heard this referred to by political scientists as 'coding' text, and it definitely falls within the domain of qualitative content analysis, but I'm yet to find anything specifically on evaluating political text. This in mind, can anyone point me towards an established metric for evaluating the political content of bodies of text? Thanks kindly, Michael Conover
you may want to talk to any number of political scientists that work in the field, there are several large digital analysis projects going on in political science that you can find by looking at the NSF grants in the field from 2006. There problem frequently faced is that it is very hard, even with normalized texts, to get more than 90% reliability in computer coding, some have managed to get up to 96%, but that 4% can really throw analysis for a loop. I don't think there is an established metric, there are contested ways of coding texts which one person or another might defend, but here: http://enapt.org/bibliography/ is a good place to start your research. On Nov 22, 2010, at 10:36 AM, Michael Conover wrote:
Our research group (made up of physicists and computer scientists) needs to evaluate the political content of short pieces of text. To this end we've constructed a rubric for classification, ranging from strong left, lean left, and neutral, to lean right, strong right, and unclassifiable. While the criteria of the rubric are rather clear cut (a 'strong' classified sentence would contain attacks on a person or group's character, hyperbolic language, or intense / fanatical expressions of support for an issue), we all suspect that this is a solved problem.
I've heard this referred to by political scientists as 'coding' text, and it definitely falls within the domain of qualitative content analysis, but I'm yet to find anything specifically on evaluating political text. This in mind, can anyone point me towards an established metric for evaluating the political content of bodies of text?
Thanks kindly, Michael Conover _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
Jeremy Hunsinger Center for Digital Discourse and Culture Political Science Virginia Tech Everything you can imagine is real. --Pablo Picasso
Hi Michael, The work of linguists who work within the theoretical and methodological framework of Critical Discourse Analysis might be of some use to you. A starting point: Fairclough, Norman (2001) Language and Power (Second Edition). ed.Candlin, Christopher N., Second Edition edn., Language in Social Life Series, Harlow: Longman. Fairclough, Norman (1995) Critical Discourse Analysis: The Critical Study of Language. ed.Candlin, Christopher N., Language in Social Life Series, Harlow: Pearson. Wodak, Ruth and Michael Meyer (eds.) (2009) Methods of Critical Discourse Analysis (Second Edition). London: Sage. Best, Barb -- Barbara Clark, PhD student School of Languages, Linguistics and Film Queen Mary, University of London Mile End Road London E1 4NS United Kingdom b.l.clark@qmul.ac.uk mobile: (07912) 690075 On 22 Nov 2010, at 15:36, Michael Conover wrote:
Our research group (made up of physicists and computer scientists) needs to evaluate the political content of short pieces of text. To this end we've constructed a rubric for classification, ranging from strong left, lean left, and neutral, to lean right, strong right, and unclassifiable. While the criteria of the rubric are rather clear cut (a 'strong' classified sentence would contain attacks on a person or group's character, hyperbolic language, or intense / fanatical expressions of support for an issue), we all suspect that this is a solved problem.
I've heard this referred to by political scientists as 'coding' text, and it definitely falls within the domain of qualitative content analysis, but I'm yet to find anything specifically on evaluating political text. This in mind, can anyone point me towards an established metric for evaluating the political content of bodies of text?
Thanks kindly, Michael Conover _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
Michael, Coding political characteristics is a common task in the field of political communication. There are no broadly established metrics as far as I am aware, but here are a few recent references: * Freelon, D. G. (2010). Analyzing online political discussion using three models of democratic communication. New Media & Society, 12(7), 1172-1190. * Janssen, D., & Kies, R. (2005). Online forums and deliberative democracy. Acta Politica, 40(3), 317-335. * Kelly, J., Fisher, D., & Smith, M. (2005). Debate, division, and diversity: Political discourse networks in USENET newsgroups. Online Deliberation 2005/DIAC-2005. * Papacharissi, Z. (2004). Democracy online: Civility, politeness, and the democratic potential of online political discussion groups. New Media & Society, 6(2), 259-283. * Stromer-Galley, J. (2007). Measuring deliberation's content: A coding scheme. Journal of Public Deliberation, 3(1), 1-37. When coding your texts, you will want to follow standard content analysis practices, one of which is the assessment of intercoder reliability. In this you may find my online intercoder reliability application ReCal of some value: http://dfreelon.org/utils/recalfront/ . It calculates not only percent agreement but also chance-corrected reliability coefficients such as Scott's pi, Cohen's kappa, and Krippendorff's alpha. Hope some of this helps, ~DEEN On 11/22/10 10:36 AM, Michael Conover wrote:
Our research group (made up of physicists and computer scientists) needs to evaluate the political content of short pieces of text. To this end we've constructed a rubric for classification, ranging from strong left, lean left, and neutral, to lean right, strong right, and unclassifiable. While the criteria of the rubric are rather clear cut (a 'strong' classified sentence would contain attacks on a person or group's character, hyperbolic language, or intense / fanatical expressions of support for an issue), we all suspect that this is a solved problem.
I've heard this referred to by political scientists as 'coding' text, and it definitely falls within the domain of qualitative content analysis, but I'm yet to find anything specifically on evaluating political text. This in mind, can anyone point me towards an established metric for evaluating the political content of bodies of text?
Thanks kindly, Michael Conover _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-- Deen Freelon Ph.D. Candidate, Dept. of Communication University of Washington dfreelon@uw.edu http://dfreelon.org/
participants (4)
-
Barbara Clark -
Deen Freelon -
jeremy hunsinger -
Michael Conover