I started writing and publishing scholarly articles in the early 2000s, shortly after fully embracing the digital era. Somewhat of a laggard on both fronts – completing postsecondary education and technological adoption – I found that my introduction to academic peer review and internet culture happened simultaneously.
My very first research article, which I delivered in person to the managing editor in hard-copy form, meandered through a then paper-based review process. It was returned by regular mail with a rejection notice based on “Reviewer B’s” scant comments, which included incomplete sentences full of spelling mistakes and harsh language. Inexperienced at the time, I imagined that the comments were written by a far busier senior colleague who knew something I did not and had very little time to fully justify their observations and decision.
Fifteen years and several published articles later, my ongoing participation in academic peer review has paralleled the proliferation of cyberbullying and troll culture that remains a serious problem in other parts of the internet. Recently, a colleague approached me with concerns about reviews they received on their first attempt at a journal article. This time, Reviewer B went so far as to make assumptions about the authors’ ethnicity and made derogatory comments about the relationship between junior and senior authors. Rather than engage with the content of the paper and critique the analysis, they offered opinions based on their own agenda.
These “reviews,” although 15 years apart, shared common traits: an overall negative tone; snide, off-point comments based on erroneous assumptions and personal bias; and vague criticisms that lacked connections with those offered by other reviewers or editors. Finally, the reviews were hypocritical. That is, they could be criticized for the same issues they were condemning: unsupported claims, sloppiness and brevity. They provided nothing constructive to work on, indicating that the troll reviewer didn’t care about their role, the journal that provided them the service opportunity, nor their supposed colleagues – their counterparts in the review process.
The experience leaves me asking: what is the value of academic peer review in the social sciences? The process is intended to subject research to objective or at least arm’s-length inspection by other specialists. It is supposed to encourage work that meets high standards and thus provide assurances to readers and knowledge users that gratuitous claims and unacceptable analyses are not published without close review by qualified people. Review processes follow established protocols to guide criticism of content but also to structure the conduct of academic exchange.
After much consideration, I believe that current peer-review processes often do the opposite of what they are supposed to achieve. That is, the double-blind system provides no assurance to me – and, by extension, the readers – that reviewers are qualified professionals. The blind review process can reduce accountability, leading to poor-quality service work, which reviewers can no doubt still count on in their annual evaluations without having properly served their journal, their discipline or their colleagues. The blinded format can encourage boorish and unethical behaviour. To top it off, troll reviews waste an incredible amount of professional time and energy of authors, editors and colleagues who reviewed earlier drafts.
This experience has left me with many questions:
- Would open reviews improve collegiality and accountability?
- Do digital formats offer ways to tighten feedback between and among authors, reviewers and editors that can improve the quality and efficiency of peer-review (e.g., using online audiovisual platforms to critique and defend papers)?
- More broadly, if I am receiving these kinds of reviews from supposed colleagues who are members of a discipline, association or network of which I am part, do I really want to participate?
- What are some other ways to vet research, confirm soundness and quality, and communicate with our peers?
Now from a place of modest scholarly experience, I see that the internet and social media have simply enabled troll behaviour that has existed in academic peer review all along. Digital advances may have increased efficiency and the volume of papers in review; however, improving author-reviewer interactions so that they bolster knowledge still requires rethinking the purpose and format of peer review. As we continue to weigh advances in technology to improve review delivery options, it’s high time we scrutinize the academic peer-review process itself.
Ryan Bullock is an associate professor of environmental studies and director of the Centre for Forest Interdisciplinary Research at the University of Winnipeg.
I long ago gave up blind submissions to journals where I am not known. Fortunately for me, there are many other ways to publish strong work (outside of the sciences and social sciences, at least). Open review is the answer, I think: both the author and the reviewers should know each other’s identity. That is the only way to ensure real accountability. Many European publishers used to employ a similar process, though “double-blind” review has been forced on them by the administrative needs of faculty members in the Anglosphere. And BTW, I started submitting articles well before the internet age and reviewers’ comments were often just as irrational, bloody-minded, uninformed, misguided, belligerent, disdainful, disrespectful and downright ornery then as now!
I agree with Andrew Gow’s comments, and have had similar experiences. One SSHRC project was (narrowly) rejected I believe on the basis of one extremely negative review that included sentence fragments, many exclamation points, and very little reference to what the project was about.. Since the other review was very positive, i can only assume that the committee, needing a reason to eliminate some proposals, responded to the negative review without fully endorsing it.
I have also had excellent experience with reviewing and being reviewed by journals with an open review process.
Blind reviews are often just fine –written by reasonable and responsible people who contribute to scholarship. But I have also seen some humdingers. I doubt sane people would put their names on them. The fetish for blind reviews needs to be held up to some serious scrutiny. It is a technique, and only a technique, and I don’t think it works.
I have been publishing for far longer, in the life and earth sciences; first paper in 1987. As an author I too have seen much boorish, arrogant, and hypocritical reviewing, and as a reviewer have seem equally arrogant, boorish and hypocritical responses by authors to my reviews. I have also seen both very ethical and credible reviews and reviewing as an associate editor for a journal, and some truly good handling of poor review practices by editors. However, I have also seen editors ‘drop the ball’ and let bad review practices happen, and have resorted to calling them on these lapses – not always with success.
As a consequence, I do think the blind review (‘anonymous review’ as its more commonly called in the physical sciences) has had its day.
Some European journals, such as those published by Copernicus Press (primarily in English), follow an open review model where anyone in the discipline can post comments on an article under review (published first online as a Discussions paper), and the 2-3 official reviewers also have their reviews published online, along with the authors’ responses. From what I have seen of these – as both an author and a reviewer – is that the informal and formal reviews are collegial, constructive and helpful to the authors. And as far as I can tell, there is no diminishment in quality, and papers in these journals command high citation rates and other impact metrics. For my graduate students, they say this is a far more attractive opportunity for publishing, and a far more positive experience.
Ryan. I agree with you. I have written along these lines in University Affairs: “Time to rethink peer review
Evaluating scholarly work in the Internet age.” By MURRAY DINEEN | DEC 05 2012. While my interests are those of the humanities and the arts, I believe it is particularly important for the sciences and for legal and political studies, where research must be evaluated fairly and in a timely fashion. I will conclude simply by pasting an excerpt of my thoughts as published in 2012, and draw your attention to the work of Rosanna Tamburri, from 2012, cited in my article.
“The Internet allows for timely and humane forms of exchange in scholarship. In the hands of an editor, peer review could become a form of colloquy, an exchange between author and reviewers. “Open peer review” and “open peer commentary” should become fully accepted practices of scholarly review.”
In four decades of submission of scholarly work, I have never received the sort of snide review that Ryan Bullock mentions. Nor have I ever written this sort of review. Anyone receiving such a review should complain about its content to the editor who selected this reviewer, attaching a copy of the review. Any editor worth their salt would immediately stop using such a reviewer, and would disount their recommendation.
That is theory. In practice it does not work that way.
The reports can be so devious and underhanded that you even do not find the words to answer.
If you just tell the truth it turns the tables on you. You have never been bullied.