Re: [Air-l] validity of the research
Aurelija raises a question that echoes the "paradigm dialogue" over the past quarter century--debates among qualitative and quantitative researchers about what counts for truth, what it means to know. [Up on soapbox] Questions of validity are fundamental to the logic of your research, which points to what you're asking and how you set about to answer it. There's no one right answer (i.e., validity = "triangulate"); validity is the mortar of your overall approach. There are also different kinds of threats to validity, ranging across whether you captured what you thought you captured to whether you've considered alternative explanations to understand what you found. [Down from soapbox] "...just trust..." isn't enough, but what _is_ ? Especially when you've given up on notions of objective truth, as I assume you have if you're doing some kind of interpretive analysis of online exchanges. Here's what educational researcher Joe Maxwell has to say: "[T]he idea of objective truth isn't essential to a theory of validity that does what most researchers want it to do, which is to give them some grounds for distinguishing accounts that are credible from those that are not." (Qualitative Research Design, p. 87). So then, what's credible? The answer lies in the integrity of your design--how different facets--theory (and the epistemology that implies) and methodology--come together to form a whole. David -------------------air-l-admin@aoir.org------------------- To:air-l@aoir.org CC: Subject:[Air-l] validity of the research hello :) Introducing qualitative research in the seminar, I was asked a question about the validity of online researches: how may we trust the results, when we don't get non-verbal information and how can we be sure whether the respondent says the truth or lies... I usually use additional methods for validation, such as observation or interviews IRL, but in some cases I (and possibly other researchers too) just trust, what my respondents say and build my conclusions on their words... so I wanted to ask, how do you deal with the question of validity doing the research online? Regards Aurelija Dagilyte _______________________________________________ Air-l mailing list Air-l@aoir.org http://www.aoir.org/mailman/listinfo/air-l
I am very concerned with the models that continue to pervade evaluation of web-based searches via specific search engines on quantitative issues alone - although there are a few papers on qualitative measures - Below is an abstract I am presenting in Milwaukee, SSSS (Society for Social Studies in Science) on Karl Popper and the effects of positivism on web search results, with a method for locating good results by their salience rather than by volume. It's just another way to look at qualitative measures of validity, which as some have offered here as equally important to quantitative measures. Title: Testing hypothesis-testing: Taking Sir Karl Popper's erroneous trial for a spin on the World Wide Web. Sir Karl Popper (1902-1994) was the preeminent philosopher of science in the 20th century, and most serious scholars in the philosophy of science have responded in some way to his theories: his attacks on inductivism, his constructivist stance, and his integral role in the development of evolutionary epistemology. In this century, Popper�s explications of learning are increasingly claimed by educational theorists (Bailey, 2000). Educationalists can learn from Popper, who insisted that �all life is problem solving� -and that learners proceed by building tentative hypotheses to explain their world and use error elimination to refine those hypotheses (Popper, 1999). This author takes Popper's words, "[we] understate the importance of . . . the erroneous trial" (Popper, 1991:101) and applies them to searcher�s results on the WWW. The available logic (Boolean) may limit the number of results but not increase validity (Lawrence, 1999). Particularly, tradition scientific null-hypothesis tests do not work well on the Web. Either two many results are returned for sensible testing; contrarily, when no results are returned, this scarcely proves the condition as true(!). Null hypothesis methods are particularly suspect because cognitive research shows that we do not learn much from our behaviors when results are nil (Allison & Messick, 1988). Therefore, this author turns to human information processing to locate better techniques for searchers, especially searching for salience rather than eliminating error. Salience will assist WWW searchers in two cognitive processes: environmental scanning of cyberspace, and the cognitive structuring of their impressions based on 'goodness of fit' (Fiske & Taylor, 1991: 251). The author concludes that while Popper correctly recognized error as the crux of substantiating truth claims, he required more sophisticated feedback loops to bolster his theories. For one example, Norbert Weiner�s (1954) cybernetics could have provided a better mechanics for the cognitive processing of errors. Finally, better search methods (Clever project, 1999) and especially the recognition of super-salient �hits� will yield better results for searchers than error elimination methods. References Alison, S. T. and D. M. Messick 1988. �The feature-positive effect, attitude strength, and degree of perceived consensus�. Personality & Social Psychology Bulletin 14(321-241). Bailey, R. W., 2000. Education in the open society: Karl Popper and schooling. Aldershot, Hampshire, UK: Ashgate. Clever Project 1999. �Hypersearching the Web.� Scientific American (June): 54-59. Fiske, S. T. and S. E. Taylor, 1991. Social Cognition, 2nd Ed. New York: McGraw-Hill. Lawrence, S. and L. Giles 1999. �Accessibility of information on the web.� Nature 400: 107-109. Popper, K. 1991, �How I became a philosopher� In: Popper, K., 1999, All life is problem solving. London: Routledge. Popper, K., 1999. All life is problem solving. London: Routledge. Wiener, N. 1954. The human use of human beings : cybernetics and society. Boston, MA: Houghton Mifflin. ===== "it's easier to use your mouse than your brain" Denise Rall, Sustainable Forestry Mentoring Coordinator & PhD student, School of Education, Southern Cross University, PO Box 157, Lismore, NSW, 2480 Australia Phone +61-2-6624-8627 Fax +61-2-6624-8637 Office (Tuesdays) (02) 6620 3577 Mob 0438 233 344 http://www.scu.edu.au/schools/edu/research/deniserall/index.html __________________________________________________ Do You Yahoo!? HotJobs - Search Thousands of New Jobs http://www.hotjobs.com
participants (2)
-
david_eddy_spicer＠harvard.edu -
Denise N. Rall