Other articles that address this issue are Berinsky, Huber, and Lenz 2012 (Political Analysis) and Chandler, Mueller, and Paolacci 2013 (Behavioral Research Methods). This is a worthwhile blog post in thinking through some of the issues too - http://www.culturalcognition.net/blog/2013/7/10/fooled-twice-shame-on-who-pr... . Broadly I agree with others that it depends on the question you're asking. On Wed, Jan 8, 2014 at 8:09 PM, Tim Muntinga <munt.tim@gmail.com> wrote:
Additionally, keep in mind the nature of Mechanical Turk; is there an incentive for the specific group you are targeting to deliver high quality entries? With the amounts of projects executed on a weekly basis, problems of satisficing and routine knowledge of internal tests occur. I find this for surveys quite an important point to consider.
See for more: Kapelner, A., & Chandler, D. (2010). Preventing Satisficing in online surveys.
*----* Tim *W: *www.timmuntinga.com
2014/1/9 Aaron S. Veenstra <aaron@etchouse.com>
If you're doing experiments, MTurk is great, especially if your likely alternative is a sample of undergrads. If you're doing surveys, it's not. The U.S. users are not representative of the American population, and I'd imagine the same is true of other countries' MTurk populations. And even if it were, there's no way to prove that to any kind of satisfaction. Basically, using it for a survey that you want to generalize to the population isn't getting you a sample that's any better than posting the survey to Facebook and Twitter and asking people to spread it around.
FWIW, the skew I've seen with MTurk samples compared to the U.S. population is a) slightly too male, b) slightly too white, c) too young (though older and with more age variance than an undergrad sample), d) average income too low, and e) too liberal (though, again, less skew than I'd expect from undergrads).
Aaron
On Wed, Jan 8, 2014 at 6:40 PM, Robinson,Cory <cory.Robinson@colostate.edu> wrote:
While Qualtrics and SurveyMonkey seem to be generally well respected, they also come with hefty price tags for participant recruitment ($5-10 per American respondent). Amazon’s Mechanical Turk (aka Mturk), on the other hand, can recruit participants for far less (<$1/respondent).
What is the AoIR consensus on utilizing Mturk? I’ve seen articles both for and against using the service.
Thanks in advance for all opinions/insights.
- Cory Robinson -- Stephen Cory Robinson cory.robinson@colostate.edu<mailto:cory.robinson@colostate.edu> Office: Clark C258A http://colostate.academia.edu/StephenCoryRobinson
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
-- Aaron S. Veenstra Assistant Professor, Southern Illinois University Carbondale School of Journalism || 1232 Comm Building asveenstra@siu.edu || manytoomany.com _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/
_______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/