Hi, all. I wanted to mention that another of the organizations I'm active in, the Alliance of Digital Humanities Organizations, which puts on the annual DH conference, recently experienced some growing pains around its conference submission and reviewing process. The conference has recently seen a dramatic uptick in submissions, from a much more diverse range of fields, methodologies, and parts of the world than in the past, and as a result recent conference program processes were beset by challenges in getting the right reviewers for the right proposals. And this produced a lot of hard feelings across the board: among authors who felt rudely treated, among reviewers who felt overtasked, and among program committee members who totally *were* overtasked. This year's program committee, led by Bethany Nowviskie, introduced a number of changes in the process, which Bethany detailed in a blog post: http://nowviskie.org/2012/cats-and-ships/. The most important of these changes may have been the bidding process, in which reviewers get to request particular abstracts to review (as well as marking those abstracts for which they are not qualified). I have served as a reviewer for the last several years and can say that the proposals I was asked to review this year were far more appropriate to my subfield than they had ever before been. The process also included a few other crucial changes: First, after a reviewer submitted her reviews, she was able to see the other reviews of those abstracts (though without reviewer names attached), and she could modify or add to her reviews in response. Second, after the review period was closed, authors were given the opportunity to respond to the reviews. And finally, the program committee was able to return to particular reviewers to ask them for clarification or reconsideration, before making a final decision. All of this, from what I've heard, made the committee's process more complex, but I did not see any complaints online about the process or its results this year, while recent years had produced lots of audible discontent. Bethany has promised to write some more assessing the results of the process; the conference is coming up, so perhaps that will be available soon. All best, Kathleen -- Kathleen Fitzpatrick // Director of Scholarly Communication Modern Language Association // mla.org // @kfitz On Thu, May 30, 2013 at 2:00 PM, Holly Kruse <holly.kruse@gmail.com> wrote:
I think that it's great that we're having this discusson on the list. I've reviewed for the past eight or nine AoIR conferences, and I like to think that I have a pretty good handle on reviewing for this conference. I could be wrong. Still, I propose that for the next conference we craft clear guidelines for reviewers, so that reviewers are less apt, for instance, to expect a roundtable abstract to demonstrate the same level of theoretical development as paper proposals for a panel. Likewise, we could do our best to ensure that if we keep the SPIR template for submissions, reviewers are aware that not including a methodology or results section is totally fine if it's not appropriate for the paper that's being proposed. I'm willing to spearhead this effort at clarification.
Holly _______________________________________________ The Air-L@listserv.aoir.org mailing list is provided by the Association of Internet Researchers http://aoir.org Subscribe, change options or unsubscribe at: http://listserv.aoir.org/listinfo.cgi/air-l-aoir.org
Join the Association of Internet Researchers: http://www.aoir.org/