Call for Papers A special issue of the Journal of Information Technology & Politics "Text Annotation for Political Science Research" Text is an important data source for political science research. Large, digitized text collections are becoming increasingly common. Yet most political scientists have little familiarity with the language-processing methodologies available to support research using these collections. Specifically, we are interested in methodologies from the fields of information retrieval, natural language processing, and machine learning. These techniques facilitate the automatic searching, organizing, categorizing, and extracting of information from digitized text. At a high level, the goal of language-processing is to provide one or more semantic annotations on the text. The political science question of interest can then be explored using these annotations. Text annotation techniques vary not only according to the type of semantic annotation required, but also according to the degree of manual intervention involved in the annotation process: text annotation tasks can be accomplished entirely manually ( i.e., via human content coding), entirely automatically (e.g. via keyword-based search or text clustering algorithms), automatically after a manual training period (i.e. via "supervised" machine learning methods), or semi-automatically ( e.g. via "weakly supervised" machine learning methods that acquire automatic annotation systems from very small amounts of manually labeled text). Although the potential of text annotation methods for political science research is enormous, it is understandably difficult for researchers to know where to start. In addition, in contrast to other research methodologies in the social sciences, the criteria for evaluating social science results that rely on automatic text annotation systems are not widely known, accepted, or appreciated. Keyword searches, for example, are commonly used to trace changing political attention across time, but rarely is attention given to their reliability or accuracy, raising doubts about the validity of researcher inferences. The aim of the special issue is to solicit and publish papers that provide a clear view of the state of the art in text annotation and evaluation, especially for political science. How do the techniques map onto major questions addressed by political scientists? What kinds of problems have been addressed in existing work and what text annotation methods have proven most successful? Are standard statistical measures of accuracy, recall, and precision adequate for evaluating the performance of the text annotation technique, or are new evaluation procedures needed that simultaneously consider the social science question being investigated? Given these interests, we therefore encourage submissions in the following areas: • tutorial-style surveys of state-of-the-art techniques in human language technologies and text annotation; • surveys of the state-of-the-art in the application of language-processing techniques in the social sciences, particularly in political science; • comparisons of competing text annotation methodologies on the same corpus/corpora; • innovative evaluation and diagnostic methods; • studies of text annotation methods that try to limit the amount of costly, manually annotated data for training automatic annotators, e.g. active learning; • specific applications and evaluations of language-processing and text annotation techniques; • applications of the text-processing techniques on non-social science problems that point the way to innovative social science applications; and • surveys of the available language-processing tools and resources with guidance for when to use them. All submissions must be prepared according to the submission guidelines available at: http://www.jitp.net The initial submission is due by November 1, 2007 The guest editors for the special issue are: Claire Cardie, Professor Computer Science and Information Science 4130 Upson Hall Cornell University Ithaca NY 14853-7501 cardie@cs.cornell.edu John Wilkerson, Associate Professor Department of Political Science 101 Gowen Hall University of Washington Seattle WA 98195-353530 jwilker@u.washington.edu -- Dr. Stuart W. Shulman Director, Sara Fine Institute School of Information Sciences Director, Qualitative Data Analysis Program University Center for Social and Urban Research University of Pittsburgh