david silver wrote:

ooh, interesting sounding thread!

like frank, i'm not sure if the study was an example of unethical research rather than methodologically flawed research. along those lines, in addition to the notorious rimm report (for a great summary of the controversy see http://www.fair.org/media-beat/950719.html) i'll add two more:

1. Carnegie Mellon's HomeNet study

the study: http://www-2.cs.cmu.edu/afs/cs.cmu.edu/user/copetas/www/public/pr/aug31-98.h...

reaction: http://www.columbia.edu/cu/21stC/issue-3.4/featherstone.html

I have read on a number of occasions attacks on the HomeNet studies for poor methodology. I am yet to read in these same messages details as to why considering the claims that were REALLY made by the researchers as to why the research was so "Flawed". The depression claim was not made as the journalists wrote. Nearly all research is flawed, the question then is relative. Perhaps David would like to explain why HomeNet is particularly flawed research. I should point out that I have NO CONNECTION to HomeNet although I very much liked many of the papers that have come out this study. For example M. Christ, R. Krishnan, D. Nagin, R. Kraut, O. Günther, Trajectories of individual WWW usage: implications for electronic commerce, Proc. 34th Hawaii International Conference on System Science (HICSS-34), 2001. The growing presence of the Web in everyday life is inextricably connected to the exponential growth in number and variety of Web sites offering information, commerce and services. While the number of users making use of the Internet and the Web has also grown tremendously, little is known about the extent to which individuals utilize (say, in terms of the number of visits to web sites) the Web and the trajectory of the change over time of such utilization. For example, we do not know whether the overall growth in Web usage is attributable to the increased numbers of users or to increased intensity of use of established users or both. This research is aimed to report the results of an analysis of three years of longitudinal data on residential Web usage. This data was assembled as part of the HomeNet project at Carnegie Mellon University. Drawing upon recent advances in semi-parametric, group-based statistical modeling, I examine whether there are distinctive clusters of trajectories of Web usage. I find that Web users can be clustered into four groups with distinct trajectories of use. Each of these groups achieve saturation in their extent of Web usage as measured in the number of distinct Web sites they visit over time. Demographic profiles of these different user groups will also be developed. The results are considered to have important implications for Internet marketing strategy, and public policy pertaining to the digital divide.

Quentin, i mentioned the HomeNet study because it seems to me that it qualifies as one of the major controversies in internet research that chris and christian are seeking. my point about the methodologically flawed aspect was less a critique of the methods (i'm hardly an expert on the quantitative analysis put forth by the authors) and more to the point that the controversy was methodologically-based rather than ethically-based. i could easily be wrong though and would like to hear more from others. david silver On Wed, 10 Jul 2002, Quentin (Gad) Jones wrote:

I have read on a number of occasions attacks on the HomeNet studies for poor methodology. I am yet to read in these same messages details as to why considering the claims that were REALLY made by the researchers as to why the research was so "Flawed". The depression claim was not made as the journalists wrote. Nearly all research is flawed, the question then is relative. Perhaps David would like to explain why HomeNet is particularly flawed research.

Sorry to be so late in reacting. The issue caught my interest after three weeks on a Swedish Island - Gotland. Sunny! So I may need some intellectual change! Here is my reaction to the discussion.

Thanks for putting the homenet study as a warning example! I have used it myself in a course in methodology. Useful to have flawed studies! Did you see, for example, that the web-formt of the paper does not allow you to see the figures? I had to get the original paper!

I also saw that somebody asked for the "flaws" in the study. I have seen a couple: 1) the sample of respondents is not random. This is common in psychological research, but here, the sample is probably rather biased. People did get a computer to try. How many have not already got one? Who are the people who got the computer? 2) the data are mostly based on ratings. Ratings have a tendency to get biased if they are repeated. There is something that is called "regression toward the mean". This means for instance that if you are very positive at the beginning, the average will be lower at a repeated measure, just because the average was above the (supposed) mean from the beginning. Thus, one possible bias can be the following: people were happy to get a computer. Later, they, probably got disappointed with the utility of the computer. This affects all ratings. 3) the actual "significant" differences are VERY small! You have to read the original paper. I usually do not recommend my students to talk about effects in this range at all! Good methodology and ethics go together in research, as many have proposed. We are responsible to our subjects as well as to other researchers to do as good a research as possible. Why do we else have research at all? Yvonne

1. Carnegie Mellon's HomeNet study

the study: http://www-2.cs.cmu.edu/afs/cs.cmu.edu/user/copetas/www/public/pr/aug31-98.h...

reaction: http://www.columbia.edu/cu/21stC/issue-3.4/featherstone.html

-- Yvonne Waern Professor emerita CMC-research group Computer and System Sciences Stockholm University /KTH Forum 100, S-164 40 Kista, Sweden Phone (home): +46 8 500 307 18