how to measure participation rates
As part of my study of an online community I have to measure the participation rates among participants, in a way similar to Nancy Baym's in her classic study of the r.a.t.s online community. From my participation and observation of this particular community so far, it is obvious to me that most of the messages are contributed by a small group of heavy users. However, I need to quantify it, measure it and present it as a table containing percentages of posters and posts contributed. My question is how can I do it? What is the method of measuring these rates? Does it simply consist in noting down how many messages each member writes? Prima facie that may look like an easy procedure but once starting to do it it turns out to be extremely difficult since this online community is made up of more than 20 conferences each one comprising more than 50 different topical discussions. Overall the number of messages written every week is enormous. Any ideas on how I can measure overall participatio rates of such an environment? Thank you, K. Diamantaki
I found this enormously challenging and it required a lot more backstage work than it merited paragraphs in the write up. I saved all the messages to a file and had a programmer write me a program (I think it was awk) to pull out the posters and tabulate how many each address sent. I was lucky to have good programmers willing to help out for free. Doing it manually would not have been worth it. Nancy
As part of my study of an online community I have to measure the participation rates among participants, in a way similar to Nancy Baym's in her classic study of the r.a.t.s online community. From my participation and observation of this particular community so far, it is obvious to me that most of the messages are contributed by a small group of heavy users. However, I need to quantify it, measure it and present it as a table containing percentages of posters and posts contributed. My question is how can I do it? What is the method of measuring these rates? Does it simply consist in noting down how many messages each member writes? Prima facie that may look like an easy procedure but once starting to do it it turns out to be extremely difficult since this online community is made up of more than 20 conferences each one comprising more than 50 different topical discussions. Overall the number of messages written every week is enormous. Any ideas on how I can measure overall participatio rates of such an environment?
Thank you,
K. Diamantaki
-- Nancy Baym http://www.ku.edu/home/nbaym Communication Studies, University of Kansas Bailey Hall, 1440 Jayhawk Blvd., Room 102, Lawrence, KS 66045-7574, USA Association of Internet Researchers: http://aoir.org
I agree with the Nancy's expert opinion -- this is very tedious work. A co-author and I did a similar thing in a study of an online classroom but measured participation by *sampling* from the transcripts of the online classes rather than measuring participation for the entire online conversation. This let us rule out participation rates for the formation of social structure in the class, which was pretty central to our argument. Gina Neff Postdoctoral Research Fellow Institute for Labor and Employment University of California, Los Angeles
Hi Katerina, I did a thing like that semi-manual. A bit boring maybe, but not impossible and I am definitely not a programmer. Just like Nancy did, I used one text file. I made it so that every message was on a single line, with the AUTHOR (or so) tag in front. I deleted everything that was not the author tag, and transferred the result of this to MS Excel. In Excel I sorted, and then did some counting. I even computed a Lorenz Curve. If you want to know more about how I did this, I can send you the Excel-file. It takes some hours work to figure it all out (which I have done for you now ;-) ), but once you have done that, you can easily apply it to all the discussions you want. Kind regards, Carlo Hagemann Katerina wrote:
As part of my study of an online community I have to measure the participation rates among participants, in a way similar to Nancy Baym's in her classic study of the r.a.t.s online community. From my participation and observation of this particular community so far, it is obvious to me that most of the messages are contributed by a small group of heavy users. However, I need to quantify it, measure it and present it as a table containing percentages of posters and posts contributed. My question is how can I do it? What is the method of measuring these rates? Does it simply consist in noting down how many messages each member writes? Prima facie that may look like an easy procedure but once starting to do it it turns out to be extremely difficult since this online community is made up of more than 20 conferences each one comprising more than 50 different topical discussions. Overall the number of messages written every week is enormous. Any ideas on how I can measure overall participatio rates of such an environment?
Thank you,
K. Diamantaki
Katerina, Interesting and very relevant question. As people have suggested it will depend on the format of your data: - web based discussion board - usenet - email - email to web archive all of those will require different tools for you to pull out the data you care about. If it is web based - you may want to ask the organization to give you access to the database that has this information - thus you will avoid doing a whole lot of cutting and pasting from the web to your analysis tool (probably excel). If it is maillist based - you may be able to get access to the archives - there will be specific headers you can mine to get the information you need. In any case becoming friends with a good PERL programmer is a good idea. PERL is a great language for text and data munging and often what appear to be HUGE tasks for us non-programmers can be fairly trivial for them. Look up a local PERL newsgroup in Greece and see if they can point you to the right people (they tend to be fairly friendly). As you mentioned, you will most definitely find a power-law relationship in posting patterns - this should not be news - i guess you need to perhaps explain why this is the case. You may also want to examine if the frequent posters are initiating or responding to threads - in my research on open source projects - frequent posters are always responding and rarely initiate. Here are two examples of papers where I did some similar analysis: This one looks at user-to-user technical support on Usenet for Apache web server: http://opensource.mit.edu/papers/lakhanivonhippelusersupport.pdf This one looks at how people join and specialize in an open source project: http://opensource.mit.edu/papers/rp-vonkroghspaethlakhani.pdf hth, K -- =============================================== Karim R. Lakhani MIT Sloan School of Management & The Boston Consulting Group, Strategy Practice Initiative e-mail: karim.lakhani@sloan.mit.edu | lakhani.karim@bcg.com voice: 617-851-1224 fax: 617-344-0403 http://spoudaiospaizen.net/ http://opensource.mit.edu | http://freesoftware.mit.edu http://userinnovation.mit.edu
In any case becoming friends with a good PERL programmer is a good idea. PERL is a great language for text and data munging and often what appear to be HUGE tasks for us non-programmers can be fairly trivial for them. Look up a local PERL newsgroup in Greece and see if they can point you to the right people (they tend to be fairly friendly).
here's a url with contact info for the athens, greece perl mongers (user) group: http://www.pm.org/groups/307.html
participants (6)
-
Carlo Hagemann -
elijah wright -
Gina Neff -
Karim R. Lakhani -
Katerina -
Nancy Baym