Skip navigation
IN MY OPINION

Analysis reveals smaller universities being shortchanged in NSERC competitions

Anomalies in Discovery Grant evaluations require immediate attention, say researchers.

By DOUGLAS MORRIS ET AL | JUN 06 2016

Prime Minister Trudeau’s mandate letter to Science Minister Duncan states in part that the Minister must “Examine options to strengthen the recognition of, and support for, fundamental research to support new discoveries.” The statement is a welcome recognition of the importance of fundamental research and discovery to Canada and Canadians. We applaud the implicit message that the government, through its granting councils, must reinvest in blue skies research. However, we worry that, unless there are changes, those funds may not be invested in the most effective ways to maximize Canada’s research capacity, capture the greatest diversity of new ideas, recruit the best future talent, and serve all of Canada’s regions and peoples.

This concern arises from our recently completed analyses of the Natural Sciences and Engineering Research Council of Canada’s Discovery Grant decisions. The analyses consistently reveal that the probability of success and the mean monetary value of awards in Discovery Grant competitions are higher for large universities than for small- and medium-sized universities. The funding level awarded to an application depends on which of 16 quality bins it is allocated to (typically only the top 10 bins receive grants). Lower success rates and funding levels by applicants from small and medium universities can thus occur only if their applications are typically allocated to “lower” bins than those from large universities. The data confirm our expectation: a greater proportion of applications from small- and medium-sized universities fall into lower-quality bins than is the case for large universities.

We can forecast future numbers of Discovery Grants by calculating attrition of grants held by established researchers and recruitment of newly funded researchers. Recruitment of newly funded researchers is less than the number of grants being “lost” by established researchers. This imbalance between attrition and recruitment does not bode well for Canada’s science future. If these trends continue, the number of Discovery Grants will decline across all institutions irrespective of size. Smaller universities will bear the brunt of losses.

Critics will note that Discovery funding rates and levels have always been lower at small than at large universities. They will attribute the discrepancy to effects such as differences in priorities, research culture and infrastructure, graduate programming, history, and recruitment of competitive faculty. Teaching loads are typically higher at medium and small universities, graduate training opportunities are often less, and the critical mass necessary to form integrated research teams may be missing. Each effect is likely responsible for some degree of lower Discovery Grant success in small institutions. Even so, there are at least two basic reasons to suspect that differences in funding success and grant size represent true anomalies.

First, all applications are evaluated on three equally scored metrics: excellence of the researcher, merit of the proposal and contributions to training of highly qualified personnel, or HQP. The combined score determines the quality bin allocated to each application. Virtually all first-time applicants enter the competition with similar HQP records because most lack previous opportunities to supervise and train graduate students. Consequently, the assessment of HQP for first-time applicants is typically focussed “on the plan for future training” (NSERC Discovery Grants Peer Review Manual 2015-2016, page 25). We can thereby anticipate that the assessment of HQP for first-time applicants should “level the field.” If the field is indeed level, then differences in grant funding success between small and large universities should tend to disappear for these early career applicants. Comparisons between medium and large schools confirm our intuition. There is no statistically significant difference in their likelihood of successful funding. That is not the case at small universities. Success rates are significantly lower for early career applicants from small universities relative to their peers from large institutions. The pattern is perplexing and disturbing because it suggests that scores allocated for HQP to early career researchers are higher at large than small universities. Should one expect consistently superior plans for future training from applicants at large institutions?

Second, the distortion in funding success and funding levels is self-reinforcing. Canada’s Federal Research Support Fund is linked directly to the funding received from tri-council funding agencies. Reduced grant success yields less support for research that necessarily feeds back into an institution’s research opportunities and future success.

NSERC began to address the problem in the 2015 competition by instituting a pilot program of Discovery Development Grants for deserving but unfunded Discovery Grant applicants. The program is designed to allow its recipients to maintain high-quality research and research training in small universities. But the small size of the grants ($10,000 annually), even if matched by institutional funding, and their short duration (two years), are bound to be ineffective in a system where so many better-funded and higher-ranking applicants subsequently fail in attempts to renew their Discovery Grants.

It is not exclusively the patterns and projections that concern us, but also their consequences. Smaller universities cast a wide net that identifies and encourages promising students to pursue research. Smaller universities serve our disparate regions, our rural communities and the heartland of our nation’s resource-driven economy. Those regions and communities are not well-served by concentrating fewer grant-holding professors in fewer universities. Doing so compromises Canada’s future research capacity and it reduces our ability to solve regional problems. It limits our options to understand and incorporate regional variance into ideas and policy. It also denies bright students in rural Canada opportunities to contribute to our collective futures.

We applaud the government for investing in science and engineering. The economic, social and personal well-being of Canadians depends on discovery that leads to a fair and prosperous society. But the system is not working as it should because funding anomalies limit discovery by medium- and smaller-sized universities.

Solutions to complex problems, like their causes, are manifold. The Government of Canada needs to increase investment in basic science and engineering while NSERC must resolve funding disadvantages to small- and medium-sized institutions. Those universities need to re-energize research by emphasizing its lock-step connection with high-quality teaching, allocate competitive teaching loads, and improve research infrastructure and logistical support to their vital research faculty. We encourage Minister Duncan to help in the process of removing funding anomalies so Discovery Grants can work to everyone’s advantage.

The authors are Douglas Morris, department of biology, Lakehead University; Dennis Murray, department of biology, Trent University; Hugh MacIsaac, Great Lakes Institute for Environmental Research, University of Windsor; Claude Lavoie, École supérieure d’aménagement du territoire et de développement régional, Université Laval; Peter Leavitt, department of biology, University of Regina; Michael Masson, department of psychology, University of Victoria; and Marc-Andre Villard, Département de biologie, Université de Moncton. Their full analysis, “Bias in Research Grant Evaluation Has Dire Consequences for Small Universities,” was published June 3 in the journal PLOS ONE.

COMMENTS
Post a comment
University Affairs moderates all comments according to the following guidelines. If approved, comments generally appear within one business day. We may republish particularly insightful remarks in our print edition or elsewhere.

Your email address will not be published. Required fields are marked *

  1. Philip G Hultin / June 6, 2016 at 2:46 pm

    The problem of bias based on the applicant’s university is not limited to “small” versus “large”. There is a distinct pecking order within each discipline, and departments that are at the bottom of this hierarchy suffer accordingly no matter what the overall size of the institution. The perception of quality has a lot to do with where one is located and how often one can meet face to face with one’s peers. Those of us “from the boonies” notice that the “cool kids” tend to hang out together at conferences, and it sure looks like they take care of one another when grant renewal time comes up.

    It doesn’t seem to matter what you publish or how good your students are. If you are not part of the clique, you will get pushed out of the Discovery Grant club eventually.

  2. John Robertson / June 7, 2016 at 1:10 pm

    “Should one expect consistently superior plans for future training from applicants at large institutions?”

    Why would you not?? Recall the point that large universities have greater infrastructure and a multitude of existing programs for training — these lead directly to better-defined training programs. Indeed, I’d expect a greater difference here than for the merit of the proposal.

    Larger universities have regular seminars and existing groups that junior investigators can latch onto and leverage for their proposed training programs. They have a pool of successful senior investigators to help mentor the junior ones and critique their applications. They have NSERC CREATE programs ongoing, and local industry and entrepreneurship incubators to partner with. They have institutional scholarships and travel awards to help increase the likelihood of sending trainees to international conferences even if the Discovery itself won’t be able to fund those activities. Plus equipment and facilities that can serve as unique training grounds.

    And to put a personal spin on it, larger universities have grant editors and other enablers to help with mentorship and defining research programs. While I help our junior investigators will all aspects of their first Discovery application, a part where I can make the biggest difference is in developing their HQP training plan and connecting them with the resources they need to be successful.

    • Andrew Park / June 8, 2016 at 9:21 pm

      John Robertson’s comments appear to assume that small universities have none of the infrastructure and advantages that he mentions. I am sure lots of small university professors and administrators could put him straight on that front. Small institutions also have some advantages that large universities do not have. It’s not just about graduate students you know. Honours candidates and undergraduate research assistants are HQP too. And many graduates from excellent small university programs have benefitted from small class sizes, and the personal mentorship that small institutions can provide at that level. Contrast some of the larger institutions where professors have huge stables of graduate students, assistants and post-docs.

      But I suspect (to borrow a current vernacular) that Mr. Robertson is suffering from unacknowledged “large-university priviledge” that sees the big universities as havens of all that is new and good. Institutions have it as well, since it is well known that the larger institutions have banded together to lobby aggressively for a larger share of our fairly stagnant research pie. But there is a lot of good research and training going on in Canada’s smaller institutions, and all their professors are asking are an equitable shot at winning scarce funds.

« »
--ph--