RE: [Air-l] Abstract: Measure of Discussion Activity in Threaded Discussion Spaces
Dear David, I tried your formulation and it is OK with small amount of replies. When I went to mass discussion out of the university forums, the d value is very high and the interpretation can't tell anything (what is d=13.84? how much this value is bigger than 2.4?). Additionally, Rafaeli and Jones proposed a hierarchy of interactivity (e.g. http://gsb.haifa.ac.il/~sheizaf/publications/finalGROUP99Submission.htm) that takes into account more variables. It seems relevant to your paper. Avner Caspi SOHAM The Open University of Israel 16 Klauzner St. Tel Aviv Israel -----Original Message----- From: David Wiley [mailto:dw2@opencontent.org] Sent: Thursday, August 08, 2002 7:41 PM To: air-l@aoir.org Cc: david.wiley@usu.edu Subject: [Air-l] Abstract: Measure of Discussion Activity in Threaded Discussion Spaces I've been lurking on this list long enough, I suppose. David Wiley, Dept. of Instructional Technology, Utah State University. How do you do? This paper in progress describes a manner of calculating a measure of discussion activity (*not* quality) in threaded discussion spaces. Based on an operationalization of discussion activity in which deeper reply levels suggest more intense discussion activity, a primitive formula for calculating mean reply depths for discussion archives is presented, together with methods for adjusting the measure to account for misthreaded messages and non-discussive messages in the archive. Sample calculations are shown and discussed for six month archives of two mailing lists relating to open source software. http://wiley.ed.usu.edu/docs/discussion08.pdf I would love to receive any comments anyone has on this work, especially pointers to related work. I would also be happy to share the software we used if people are interested. It currently runs the calculation on Mailman mailing list archives (and, incidentally, converts these archives into either edge lists or adjacency matrices for other kinds of analysis), though we're expanding it to handle webboards like Slashdot and kuro5hin. Theoretically it could be extended to process any archive with a consistent presentation format, as long as you aren't afraid of regular expressions <g>. Anyway, pleased to meet you. Here's to more AoIR fun, David _______________________________________________ Air-l mailing list Air-l@aoir.org http://www.aoir.org/mailman/listinfo/air-l
On Sun, 11 Aug 2002, Avner Caspi wrote:
Dear David, I tried your formulation and it is OK with small amount of replies. When I went to mass discussion out of the university forums, the d value is very high and the interpretation can't tell anything (what is d=13.84? how much this value is bigger than 2.4?).
Thanks for your feedback! I'll try to address these issues all at once. First, as for the semantics of the d value, d is the average reply depth; in other words, d simply tells you what level of reply the average message is at. So, for the tiny thread: A (Reply depth of 0) -B (Reply depth of 1) --C (Reply depth of 2) The unadjusted reply depth would be (0+1+2)/3 = 1. The paper presents some adjustments to fine tune this value, which almost always lower it. So, the difference between d=13 and d=2 is that the d=13 thread would have to look like this: A -B --C ---D <snip> -----------L ------------M -------------N --------------O --------------O --------------O with many more 14th level replies for the value to ever get over 13, while a d=2 thread looks much more normal, something like: A -B --C ---D ---D ---D ---D d=13 tells you that one group is discussing something to a much more significant depth than the group whose d=2. As for the value seeming to grow in size as the number of messages grew, this should not happen as d is intended to be an average, and should therefore be unaffected by the size of the archive. The discussion in the paper is obviously not be clear enough (which is why I asked AoIR for feedback!). I will try to fix this. Also, would you be willing to send me the archive whose d=13 so that I could run it through our software here? I would love to see it!
Additionally, Rafaeli and Jones proposed a hierarchy of interactivity (e.g. http://gsb.haifa.ac.il/~sheizaf/publications/finalGROUP99Submission.htm) that takes into account more variables. It seems relevant to your paper.
Have only just skimmed this but it looks excellent! Our goal was to create a value that provides a thumbnail of the level of discussion activity whose calculation is easily automated, so that it could be integrated into Course Management Systems and the like. It looks like this paper you've referenced could help improve our calculation. Many thanks for this pointer and your other feedback! David http://wiley.ed.usu.edu/
participants (2)
-
Avner Caspi -
David Wiley