It would be hard to find a university professor today who is not at least marginally familiar with impact factors. As public pressure to demonstrate and quantify the value of academic research increases, measurements of how often a particular journal, article or scholar has been cited in other professional publications have become critical to tenure and promotion decisions in a number of disciplines. And the trend is only accelerating.
As with any innovation, using impact factors as determinants of faculty performance has not lacked criticism. The standard rating system generally only considers citations within the first two years of an article’s publication, so essays with staying power are undervalued. The method doesn’t take into account articles that are cited because of their flaws, making some papers look far more important than they are. It doesn’t count books and book chapters, which hurts scholars who publish in the humanities and certain social sciences. And, it doesn’t recognize research exposure outside scholarly publications, penalizing authors who consciously try to increase the accessibility of their scholarship by appealing to a broader, more popular audience.
Some of these concerns have become less important due to new metrics that assess articles based on their impact within the first six months of publication, and also after five years. Google Book Search makes it easier to measure the impact of individual book chapters in major scholarly presses. And, innovations in Google Scholar have allowed us to assess the impact of an individual’s entire scholarly career, rather than just a single publication. Moreover, as David Kent explained in the University Affairs blog “The Black Hole” last fall, companies like Altmetric have developed programs that track discussions of scholarly research on Twitter, Facebook and in the media. As a result, faculty who seek to effect change in public policy now have their own means of demonstrating impact and justifying their approach to scholarship and publication.
Regrettably, even with these advances, the impact-factor movement captures only two of the three most significant audiences for faculty researchers: the academic community and the public at large. The third group, the postsecondary student body, continues to be ignored. Admittedly, in some fields, research designed specifically for classroom use is less important. Most of us won’t care whether an academic article that leads to a cure for cancer is accessible to undergraduate students; what matters is that our doctors read it and apply its findings.
But, in the humanities and in much of the social sciences, the student audience is often not just the scholar’s largest but also his or her most important. In any given year, an obscure article assigned to an introductory course in the dramatic arts could be read by hundreds, if not thousands, of students in Canada alone and could shape decisions regarding their future careers. An introductory textbook in psychology can, and likely does, affect even more.
In other words, the impact of these works, regardless of whether they add to the scholarly conversation or generate media buzz, is undeniable.
It is disappointing, but perhaps not all that surprising, that efforts to quantify the impact of academic research in the classroom – actions that would implicitly validate a scholarly agenda that focuses primarily on the in-class needs of one’s students – lag so far behind. Scholarly teaching, which can and should involve staying current with the literature in one’s field, is only beginning to be recognized as a legitimate academic contribution. And too many professors and administrators on tenure and promotions committees continue to view textbooks and edited volumes as inferior to journal articles because of their alleged failure to introduce new ideas into the professional discourse.
There are no quick fixes to this problem. There is no system in place to keep track of every reading assigned in every course syllabus. Between concerns about academic freedom (will many academics be willing to make their syllabi freely accessible to what could be a for-profit organization in order to validate research findings?) and the sheer organizational challenge, I suspect that there will be little political will to create one.
Nonetheless, advocacy groups that are committed to improving the student learning experience, like the new Teaching and Learning Canada, must recognize that the development of quantifiable accountability methods in the academy has left student-focused scholarship behind. An impact factor metric that privileges the value of those readings that are most often assigned for classroom use would be a chance to begin to catch up.
Adam Chapnick is deputy director of education at the Canadian Forces College and co-editor of International Journal: Canada’s Journal of Global Policy Analysis.