A better way for selecting effective university leaders

Given the increasing competitive pressures and financial challenges facing universities across North America, most academic administrators recognize that strong and effective leadership is a key ingredient for institutional success. Yet, while there has been a proliferation of writing on the topic of what effective academic leadership entails, relatively little scholarly attention has been paid to the selection process for academic leaders. Are there certain kinds of processes or methodologies that are more likely to result in the selection of more successful academic leaders?

Over the last decade, the procedures governing senior academic searches at York University, where I served as a dean and then as provost from 2003 to 2012, have changed considerably. The key driver behind this evolution has been a widely shared desire at the university to improve the reliability and the validity of the academic search process by imparting a greater sense of discipline and rigour.

Throughout the latter half of the 20th century, senior academic leaders at most North American universities were selected on a consensus-based, collegial model. The overarching priority in this model is to select an effective academic leader who enjoys broad support among the faculty members of the relevant academic unit. With this in mind, the selection procedure was structured to provide the opportunity for all interested faculty members to meet and interact with potential candidates and to form an opinion about their suitability for the position. In some cases, faculty members were permitted to vote on their preferred choice for president or dean. While the governing body of the university made the ultimate appointment, these faculty opinions, taken as a whole, were for all intents and purposes the key determinant in the selection process.

For example, procedures for decanal searches at York and many other universities often required the creation of a search committee, with a majority of the members elected by the faculty council and full-time faculty members. The search committee would consult with members of the relevant academic unit about criteria to guide the identification and selection of candidates. Based on that input, as well as its own deliberations, the committee approved a “position description” that included the responsibilities associated with the position and the qualifications needed in the next academic leader. The committee would consider potential candidates in light of that position description and eventually identify the top three to five candidates and invite these individuals for a campus visit.

The campus visit would involve candidates making a public presentation to the faculty (and usually also to staff and students), as well as a meeting with the committee and with other interested members of the community. Following the campus visits, those who met with, or heard from, the various candidates, were invited to provide their confidential written feedback and comments to the committee. Based on the committee members’ own assessments, as well as the confidential feedback, the committee would recommend to the president or board which candidates were deemed qualified for the position and in what rank order.

This process was intended to lead to the appointment of effective academic leaders who could promote the best interests of the faculty. But the way the process was structured made it extremely difficult for a candidate to be appointed who didn’t enjoy broad support among the faculty as a whole, even if they were otherwise extremely well qualified and would undoubtedly have been successful in the post.

Of course, it is axiomatic that any effective academic administrator must come to enjoy broad faculty support in order to lead effectively. The question, though, is whether in striving to achieve this laudable goal, the traditional model for senior academic searches gave undue weight to subjective assessments based on limited evidence, resulting in less effective leaders than would have emerged from a more evidence-based process.

Frailty of human judgment and intuition

In recent years, Dale Campbell has analyzed the comparative validity of different selection methods, and the findings suggest that personal interviews, particularly when they are unstructured, loose and discursive, are among the least effective methods for candidate selection (“Your Next Leader,” Community College Journal, June-July 2009). As Neville Bain and Bill Mabey explain (The People Advantage, Purdue University Press, 1999), unstructured interviews are “prone to biases of subjectivity and prejudice and there is evidence that judgments may be made very quickly on limited data.” They report that unstructured interviews rank only slightly ahead of references or graphology (the study of handwriting) in terms of validity, but are less predictive of job success than virtually any other method of candidate selection.

Why might this be so? We can find a ready answer in the groundbreaking work on judgment and decision making by Daniel Kahneman, Nobel Prize laureate in economics. In his 2011 book, Thinking Fast and Slow, he highlights the pervasive influence of intuitive impressions on our thoughts and behaviours: far from forming judgments in accordance with a rational model of decision making, human beings have a pervasive tendency to jump to conclusions based on limited evidence. This tendency stems from a complex series of factors, but of particular significance is a desire to achieve emotional coherence in our understanding of people and the world.

This desire for coherence suppresses ambiguity and spontaneously constructs understandings about people and situations that are as coherent as possible. This leads to what Kahneman describes as a “halo effect,” the “tendency to like (or dislike) everything about a person – including things you have not observed.” But the halo effect is “radically insensitive to both the quality and the quantity of the information that gives rise to impressions and intuitions.” When we form judgments about people or situations, we tend to ignore the possibility that we have limited or incomplete information, and instead our confidence in the accuracy of our judgment reflects the coherence or consistency of the story we are able to construct.

But, perhaps concerns over the frailty of human judgment and decision making do not apply in situations where those with a defined area of expertise are asked to apply that expertise to a defined task, such as selecting a candidate for a job. In fact, Kahneman concludes precisely the opposite: interviews of candidates for jobs or other opportunities are very much prone to bias and subjectivity. He considers the process for admitting students to medical school and concludes that conducting an interview is likely to diminish the accuracy of a selection procedure if the interviewers also make the final admissions decision. “Because interviewers are overconfident in their intuitions, they will assign too much weight to their personal impressions and too little weight to other sources of information, lowering validity.”

Indeed, numerous studies of decision making in a wide variety of circumstances have found that clinical predictions based on the subjective impressions of trained professionals are less accurate than statistical predictions made by combining a few scores or ratings according to a rule. This includes medical opinions about the longevity of cancer patients, the diagnosis of cardiac disease or the susceptibility of babies to sudden infant death syndrome. It also relates to business assessments of the likelihood that a new business will succeed or fail and predictions regarding the future success of high school students in university.

How to overcome the shortcomings and inconsistency of intuitive judgments? A substantial body of research indicates that it is preferable to use formulas derived from standardized factual questions than clinical judgment. Using this formulaic procedure will be much more likely to result in the selection of the best candidate, compared with doing what people normally do in such situations, says Kahneman, namely, “to go into the interview unprepared and to make choices by an overall intuitive judgment such as ‘I looked into his eyes and liked what I saw.’”

Implications for senior academic searches

This body of research has significant implications for the way senior academic searches should be conducted. At York University, the presidential search procedures were substantially revised by senate and the board of governors in 2005, following a study of search procedures by dean emeritus Peter Hogg and a report by the senate executive committee. The most significant change was to abolish the requirement that shortlisted candidates make a public presentation to the university senate, because the practice was found to be both superficial and easily politicized and because the process discouraged many desirable individuals from applying for the job. The new search process became entirely confidential. A broadly representative search committee makes a recommendation to the board, with a “super majority” of committee support. Following his appointment in accordance with these revised procedures in July 2007, President Mamdouh Shoukri initiated several similar changes in the procedures governing decanal searches in 2008.

York University now has substantial experience in the application of these revised search procedures, providing strong support for their wisdom and importance, especially for the decanal search. The changes fall into two categories: one, the role and function of the search committee; and two, the procedures employed by the search committee in evaluating candidates.
In decanal searches, the first big change was to eliminate public presentations by candidates and move to an entirely confidential search process. Recognizing that this increases the

ignificance and role of the search committee, the corollary is that committee members are now selected so that virtually the entire committee is elected by the different constituent groups within the relevant faculty. Since we adopted this process, York has attracted deans and senior administrators from other institutions who would not have been willing to participate in an open search process.

The tradeoff is that the search committee is almost entirely elected by various constituencies in the academic unit. Although the president holds the right to appoint the committee chair and one other member, all other members (on a committee of 11 or 12 members) are elected by the faculty. In effect, the faculty or unit is asked to trust the collective judgment of colleagues on a committee which the community itself has selected for this responsibility.

York also developed procedures intended to guide decanal search committees towards a more evidence-based, less intuitive assessment procedure. The first step is to identify the principal responsibilities or competencies that are required of a dean (or other senior academic administrator) in seven categories, including management experience and research and scholarly activity. Once a consensus is reached about the categories and competencies, these are incorporated into a checklist. Each committee member uses the checklist to review and score the curriculum vitae of potential candidates; the scoring of the CVs is shared within the committee and helps it to agree on a candidate shortlist.

The competencies are used to create a rubric for assessing short-listed candidates through the interview process. Again, committee members explain their rationale for scoring candidates as they did to their colleagues on the committee. After the discussion, each committee member has the chance to revise their scoring. The scores are collated and tabulated, resulting in a total score as well as a relative ranking for each of the finalist candidates. This ranking provides the basis for a committee recommendation to the president regarding the preferred candidate for the appointment.

This process is not intended to eliminate individual judgment regarding the suitability of the various candidates. Rather, the goal is to structure the exercise so that the judgments made by the committee are more reliable and more relevant to the pre-selected criteria associated with success in the position.

The chair, appointed by the president, generally is the provost. The chair’s role in this model is consciously structured as the guardian and champion of the integrity of the process. This means the chair’s first role is to share with committee members the challenges associated with the task of assessing the relative merits of candidates for senior academic searches, and second, to work closely with the committee on developing the competencies for the position and a rubric.

The changes, modified over a decade, were not without controversy when they were adopted, but our experience thus far shows that they provide a much sounder basis for identifying and appointing effective academic leaders. Our appointments through these new procedures have been widely regarded at the university as outstanding. Most importantly, this more evidence-based process has not in any way compromised the crucial value of ensuring broad-based faculty support for the academic leadership of the institution. In a context in which strong leadership remains a scarce and highly valued quality, these new procedures position the university to attract a cadre of academic leaders who are well equipped to guide the institution through the many significant challenges facing the postsecondary education sector in the years ahead.

Patrick Monahan is Deputy Attorney General for the Province of Ontario. Until he took up the post in November 2012, Mr. Monahan had spent 30 years at York University as a law professor, dean of law and, for the last three years, vice-president, academic, and provost.