Skip navigation
The Black Hole

New idea for driving scientific debate in your journal club: PREreview

What if scientific journal clubs discussed papers before submitting them to journals, and had a say before editors and reviewers did?

BY DAVID KENT | NOV 08 2017

The eLife folks directed me to a very interesting link the other day — an article published on eLife Labs that encouraged scientists across the world to engage with research articles before they have been peer reviewed. (For those who do not know eLife Labs, it’s the place where all the new and cool things that try to change the scientific publishing racket are brought to the fore — I encourage a regular look-see!)

Preprint servers such as arXiv and bioRxiv are all the rage these days — they get research out to the community much faster and allow researchers to stake a claim on a novel technology or piece of research to avoid getting scooped during a sometimes lengthy peer review process. I won’t discuss the merits/drawbacks of preprint servers here today, but I did want to draw our readers’ attention to an idea that emerged from the eLife Labs article. Authors Samantha Hindle and Daniela Saderi lobby for the already existing “scientific journal club” structure to be a little bolder than average…  start discussing papers before submission to journals and have a say before the editors and reviewers do!

Secondly, the article makes the excellent point that too much discussion gets locked away in the journal clubs and never sees the light of day:

“it can be frustrating when those detailed critiques remain within the walls of the journal club and are unlikely to reach the authors or the wider community.”

Indeed, I would go a step further and say that it’s not just about the paper as a standalone piece of work, but the context of the discussion as well — journal clubs aren’t simply “Is this paper good or bad?” Rather, they often stimulate tangential discussion that drives new ideas. Some of the best journal clubs I have been to talk about the actual paper for a minority of the session, instead focusing on teaching points, historical context, or new ideas that could be pursued as a result. Liberating that strand of thinking is key to moving forward. Biomedical science often lacks the in-depth secondary criticism that connects the literature that our colleagues in the humanities and social sciences are so used to — journal clubs are one of the few places where it still takes place.

Wading through some murky waters

Some of the biggest pushback I’ve had from colleagues when I discuss this idea is that the preprint articles are “not up to publication standard.” No doubt there will be some papers that suffer from poor editing, absent controls, or premature conclusions. These make it tough to wade through, but perhaps this is the whole point of getting the material out there earlier (to know how a paper stands up when submitted to a jury of peers). We are already inundated with too many articles to read, and preprints threaten to increase that burden and the hopeless attempt to “keep up” with the literature.

If enough of these PREreviews were completed, it could have numerous potential benefits:

  1. Authors fix the easy things before submission.
  2. Metrics could be established for “quality” of the paper (e.g., positive reviews lead to a ranking of some sort).
  3. Journals might be able to avoid external blind peer review (e.g., that article has been reviewed, we like it, we are going to publish/publicise it).

If the scientific community got behind this style of review in strong numbers, the possibilities are endless. At the very least there would be some incredible (and public!) discussion about scientific research and its impacts, although part of me worries about the practicalities (what if papers are never reviewed, do they become unimportant? Does posting prepublication make the minimum publishable unit problem even more acute?) — time will tell.

As with any new idea though — don’t knock it too hard until you try it…  so our journal club is going to tackle a bioRxiv paper in the coming weeks — I’ll be sure to report back!

 

ABOUT DAVID KENT
David Kent
Dr. David Kent is a principal investigator at the York Biomedical Research Institute at the University of York, York, UK. He trained at Western University and the University of British Columbia before spending 10 years at the University of Cambridge, UK where he ran his research group until 2019. His laboratory's research focuses on the fundamental biology of blood stem cells and how changes in their regulation lead to cancers. David has a long history of public engagement and outreach including the creation of The Black Hole in 2009.
COMMENTS
Post a comment
University Affairs moderates all comments according to the following guidelines. If approved, comments generally appear within one business day. We may republish particularly insightful remarks in our print edition or elsewhere.

Your email address will not be published. Required fields are marked *

  1. Rowland Lorimer / November 15, 2017 at 15:26

    I’ve never heard of “journal clubs” before. It strikes me, as a professor emeritus and journal editor that journal clubs of retirees with a penchant for aiding colleagues might play a very useful role in reviewing preprints or even submitted manuscripts. I know that my journal, Scholarly and Research Communication, would be happy to see something like that emerge and would facilitate passing on comments to authors.

Click to fill out a quick survey