My office mate and I were discussing the apparent randomness of grant assessment the other day. We traded stories of amazing grants that got shafted and horrible grants that somehow got cut a cheque for hundreds of thousands of dollars. In academia, we all have these tales, but funnily enough we don’t always agree on what “amazing” and “horrible” are. So how can we possibly improve the situation?
In the face of such subjectivity, we wondered (we are scientists after all!) whether we could objectify the problem – could we create a black and white decision in the world of grant writing? The result of our chat – fake grants.
What would it look like?
The fake grant would need to be believable (excellent even), but would also need to suffer from some fatal flaws that should be obvious to someone who reads the grant properly. It could range from simple things (done by others, but not mentioned) to technical things (100-fold more of something in an experimental protocol) to conceptual inconsistencies (If A happens then B must be true while also saying somewhere else if A happens, B must not be true). One can imagine dozens of ways to sabotage a grant and a few trial runs should produce something that strikes the correct balance between believability and critical mistakes.
How would it work?
This would depend on the current peer review system for granting agencies. In the case of smaller agencies where reviewers only receive one or two grants to assess, it doesn’t seem a particularly efficient way to introduce fake grants. Rather, it would seem more useful to roll this out in national and international granting agencies where the impact and utility of fake grants could be assessed systematically. As a reviewer, you could imagine receiving eight or 10 proposals, one of which might be a fake grant. You change nothing about the way you review, but perhaps an option to identify a “fake grant” might be included to fully assess the utility of the system (side bonus question: how many real grants would get flagged as fake?).
Big brother is watching
Just like we all slow down when driving past a police car or a speed camera, the fear of being caught can be quite effective in altering behaviour. If you thought that one of the 10 grants in your pile might be a fake grant, you wouldn’t want to miss it. In the first instance, fake grants would need to be reasonably common to instill this awareness. After this initial phase, the number of fake grants could be brought down so as to not waste too much reviewer time.
The tricky part: finding volunteers
Sadly, unless reviewers were blinded to researcher identity, it would be nearly impossible to also have a fake applicant. Academics are very findable people (websites, papers, etc) and research fields are often so small that you would know that the person didn’t really exist. Blinding grants would be an excellent solution, but in the absence of that, each agency would need a few heavy hitting volunteers to agree to write a fake grant – the backbone of the grant could be largely similar to others they have written (or perhaps submitted the year prior) but key changes would need to be made to introduce the fatal flaws discussed above. The ideal volunteers would be senior established researchers who would typically be inclined to receive favourable reviews based on track record and research environment.
At worst, this relatively low cost, easy-to-roll-out idea could confirm that a grant review process is robust and all fake grants are identified and dismissed – it might annoy some reviewers to know that they invested time into a fake grant, but this would be a small price to pay. At best, this process could help to identify core problems in the review process – why and how do some grants make it through despite fatal flaws? Once we understand the problem, we can better design strategies to fix the problem – fake grants might just be an easy way to start.