Analysis reveals smaller universities being shortchanged in NSERC competitions

Prime Minister Trudeau’s mandate letter to Science Minister Duncan states in part that the Minister must “Examine options to strengthen the recognition of, and support for, fundamental research to support new discoveries.” The statement is a welcome recognition of the importance of fundamental research and discovery to Canada and Canadians. We applaud the implicit message that the government, through its granting councils, must reinvest in blue skies research. However, we worry that, unless there are changes, those funds may not be invested in the most effective ways to maximize Canada’s research capacity, capture the greatest diversity of new ideas, recruit the best future talent, and serve all of Canada’s regions and peoples.

This concern arises from our recently completed analyses of the Natural Sciences and Engineering Research Council of Canada’s Discovery Grant decisions. The analyses consistently reveal that the probability of success and the mean monetary value of awards in Discovery Grant competitions are higher for large universities than for small- and medium-sized universities. The funding level awarded to an application depends on which of 16 quality bins it is allocated to (typically only the top 10 bins receive grants). Lower success rates and funding levels by applicants from small and medium universities can thus occur only if their applications are typically allocated to “lower” bins than those from large universities. The data confirm our expectation: a greater proportion of applications from small- and medium-sized universities fall into lower-quality bins than is the case for large universities.

We can forecast future numbers of Discovery Grants by calculating attrition of grants held by established researchers and recruitment of newly funded researchers. Recruitment of newly funded researchers is less than the number of grants being “lost” by established researchers. This imbalance between attrition and recruitment does not bode well for Canada’s science future. If these trends continue, the number of Discovery Grants will decline across all institutions irrespective of size. Smaller universities will bear the brunt of losses.

Critics will note that Discovery funding rates and levels have always been lower at small than at large universities. They will attribute the discrepancy to effects such as differences in priorities, research culture and infrastructure, graduate programming, history, and recruitment of competitive faculty. Teaching loads are typically higher at medium and small universities, graduate training opportunities are often less, and the critical mass necessary to form integrated research teams may be missing. Each effect is likely responsible for some degree of lower Discovery Grant success in small institutions. Even so, there are at least two basic reasons to suspect that differences in funding success and grant size represent true anomalies.

First, all applications are evaluated on three equally scored metrics: excellence of the researcher, merit of the proposal and contributions to training of highly qualified personnel, or HQP. The combined score determines the quality bin allocated to each application. Virtually all first-time applicants enter the competition with similar HQP records because most lack previous opportunities to supervise and train graduate students. Consequently, the assessment of HQP for first-time applicants is typically focussed “on the plan for future training” (NSERC Discovery Grants Peer Review Manual 2015-2016, page 25). We can thereby anticipate that the assessment of HQP for first-time applicants should “level the field.” If the field is indeed level, then differences in grant funding success between small and large universities should tend to disappear for these early career applicants. Comparisons between medium and large schools confirm our intuition. There is no statistically significant difference in their likelihood of successful funding. That is not the case at small universities. Success rates are significantly lower for early career applicants from small universities relative to their peers from large institutions. The pattern is perplexing and disturbing because it suggests that scores allocated for HQP to early career researchers are higher at large than small universities. Should one expect consistently superior plans for future training from applicants at large institutions?

Second, the distortion in funding success and funding levels is self-reinforcing. Canada’s Federal Research Support Fund is linked directly to the funding received from tri-council funding agencies. Reduced grant success yields less support for research that necessarily feeds back into an institution’s research opportunities and future success.

NSERC began to address the problem in the 2015 competition by instituting a pilot program of Discovery Development Grants for deserving but unfunded Discovery Grant applicants. The program is designed to allow its recipients to maintain high-quality research and research training in small universities. But the small size of the grants ($10,000 annually), even if matched by institutional funding, and their short duration (two years), are bound to be ineffective in a system where so many better-funded and higher-ranking applicants subsequently fail in attempts to renew their Discovery Grants.

It is not exclusively the patterns and projections that concern us, but also their consequences. Smaller universities cast a wide net that identifies and encourages promising students to pursue research. Smaller universities serve our disparate regions, our rural communities and the heartland of our nation’s resource-driven economy. Those regions and communities are not well-served by concentrating fewer grant-holding professors in fewer universities. Doing so compromises Canada’s future research capacity and it reduces our ability to solve regional problems. It limits our options to understand and incorporate regional variance into ideas and policy. It also denies bright students in rural Canada opportunities to contribute to our collective futures.

We applaud the government for investing in science and engineering. The economic, social and personal well-being of Canadians depends on discovery that leads to a fair and prosperous society. But the system is not working as it should because funding anomalies limit discovery by medium- and smaller-sized universities.

Solutions to complex problems, like their causes, are manifold. The Government of Canada needs to increase investment in basic science and engineering while NSERC must resolve funding disadvantages to small- and medium-sized institutions. Those universities need to re-energize research by emphasizing its lock-step connection with high-quality teaching, allocate competitive teaching loads, and improve research infrastructure and logistical support to their vital research faculty. We encourage Minister Duncan to help in the process of removing funding anomalies so Discovery Grants can work to everyone’s advantage.

The authors are Douglas Morris, department of biology, Lakehead University; Dennis Murray, department of biology, Trent University; Hugh MacIsaac, Great Lakes Institute for Environmental Research, University of Windsor; Claude Lavoie, École supérieure d’aménagement du territoire et de développement régional, Université Laval; Peter Leavitt, department of biology, University of Regina; Michael Masson, department of psychology, University of Victoria; and Marc-Andre Villard, Département de biologie, Université de Moncton. Their full analysis, “Bias in Research Grant Evaluation Has Dire Consequences for Small Universities,” was published June 3 in the journal PLOS ONE.