Skip navigation
The Black Hole

Grumpy old men (and women) – feedback on proposed CIHR reforms

BY DAVID KENT | AUG 10 2012

For those who have not yet heard, the CIHR plans to make major changes to their funding mechanism for health scientists. Last week, at the height of summer vacation, the CIHR released a “What CIHR heard” document that summarizes the feedback they received on the proposed changes.

At first, I was simply going to pick out what I thought were the items of highest relevance to early career researchers, but as I perused the document, I noticed something quite interesting. At the end of every survey question, the CIHR lists the group of researchers (organized by field and seniority) who agree or disagree most strongly with each of the proposed changes and in every instance of disagreement one category (out of 12) of researchers is always listed: Senior Biomedical (Basic) Researchers. It might be unfair to suggest it, but as this represents the most entrenched set of researchers (those who have been there the longest and are within the biggest CIHR research tier) it stands to reason that maybe, just maybe, they are trying to protect the system that keeps them at the top with the most resources.

Below I have pulled out the levels of disagreement based on proposed change:

  • Integrated Knowledge Translation (46%)
  • Multi-phased competition process (57%)
  • Application Focused Review (25%)
  • Use of Structured Review Criteria (35%)
  • Remote Screening Process (49%)
  • College of Reviewers (37%)

Overall, the picture is quite different, with many groups (especially in younger researchers and those in applied research areas) showing much stronger support for the proposed reforms. It seems to me that those who want things to change the most are those who do not have access to as many resources and hope that CIHR’s reforms will help redistribute the wealth.

In the very comprehensive report, numerous ideas, complaints, and comments were provided, some of which I would like to highlight:

Too many eggs in one basket – the proposed structure states that if an investigator has a program grant already, they cannot apply for a second grant (program or project). This prompted many concerns about what happens in the case of poor renewal rates (especially in the transition period from multiple grants to single grants) and whether or not it would stymie collaborative efforts between labs that each already have a program grant (i.e., they would be unable to apply for a joint grant). This is one of the main points we raised in our analysis as well and I think CIHR needs to think hard on the best solution to this concern.

Incentives – some respondents thought that providing incentives for reviewers would encourage more qualified reviewers to partake. These included honoraria, deadline extensions on one’s own applications, and increased value/duration of one’s own grant. Personally, I do not think that incentives are the way to go – it seems to me that the privileges associated with reviewing the country’s best research and the interactions with other reviewers are worth it. However, I could support the penalization of investigators who do not participate in peer review (i.e., ineligibility to apply for future grants, limited access to funding, etc).

Fear of the multi-phased process – one of the items that we most strongly supported was the blinded first stage of project grants which aims to cull poor proposals even if they come from excellent scientists. If you write a poor proposal, you don’t really deserve to get funded, it doesn’t matter who you are. It seems that support was not as strong amongst respondents to the CIHR – with respondents registering concerns about inability to assess anonymously and increased re-submission of proposals that didn’t make the cut – I wonder where our readers stand?

Many hands make the labour light – another item that provoked some concern was reviewer fatigue from having more reviewers (5-8) on all applications. Despite these assessments being intended to take a shorter amount of time, some respondents believed that in order to do a proper job, less time per application would lead to poor decisions. I disagree. To emphasize this point I will draw an analogy that I think scientists should understand. If presented with the following two sets of data, which one do you have the most confidence in?

  1. A well-respected, well resourced lab publishes a very detailed study that identifies Protein X as a marker of advanced colon cancer.
  2. 10 distinct groups from different institutions across the world all identify Protein Y as a marker of advanced colon cancer.  The data in each individual study are not as detailed or exhaustive as the first study.

If you have to choose one to add into your diagnostic tool for colon cancer, which do you choose? Does repeatability outweigh the strength of the first study?

This seems to be the reason for increasing the number of reviewers per application. The CIHR is asking for less detailed, less rigorous reviews of a less comprehensive proposal. In the event that one reviewer misses the point or over-hypes an application, their impact is curbed by additional reviewers. If a grant gets through everybody, then chances are, it has some merit. The subsequent in-depth review is meant to clean up the remaining issues and fund or not fund based on merit.

Overall, the CIHR has a lot to think about. The report they produced does not suggest much about what the outcomes will be, but does say that final decisions will be released in the autumn. We’ll stay tuned and be sure to inform our readers of its release.

 

ABOUT DAVID KENT
David Kent
Dr. David Kent is a principal investigator at the York Biomedical Research Institute at the University of York, York, UK. He trained at Western University and the University of British Columbia before spending 10 years at the University of Cambridge, UK where he ran his research group until 2019. His laboratory's research focuses on the fundamental biology of blood stem cells and how changes in their regulation lead to cancers. David has a long history of public engagement and outreach including the creation of The Black Hole in 2009.
COMMENTS
Post a comment
University Affairs moderates all comments according to the following guidelines. If approved, comments generally appear within one business day. We may republish particularly insightful remarks in our print edition or elsewhere.

Your email address will not be published.

  1. CS / August 10, 2012 at 21:33

    If only the senior biomedical guys are criticizing it, then CIHR must be doing something right! I like all the changes mentioned, except maybe the ‘new’ KT proposal (KT seems to be the lovechild of BS and politics, so no interest).

  2. PW / August 13, 2012 at 20:00

    Totally disagree with CS. KT definitely has a place. Whether it should be integrated in all programs (i.e. basic/biomedical) I’m not sure. But for all our talk of medical and health interventions and discovery, I would like to see some big push for implementing these things. Certainly no use discovering something useful but for reasons like education of health care providers or policy makers the something useful gets shelved. Also, yes, basic research into discovering “cures” are absolutely necessary, but how about helping those who are currently burdened without a cure.