TRaCE was a one-year pilot project, carried out from fall 2015 to fall 2016, about humanities PhD outcomes. Twenty-four Canadian universities took part in an experiment in data gathering about the jobs PhDs are doing, getting grads to tell their stories, connecting doctoral students with PhD graduates inside and outside the academy, and inviting grads to serve as mentors, presenters, guest teachers, and so on, in the graduate programs they’d left behind. TRaCE has the potential to become a valuable resource for those in PhD programs and those thinking about doing a PhD, as well as for educators, employers and policy-makers.
The people who worked on TRaCE included a planning group with members from across the country and several data experts, and a head office group at McGill University. Both groups were a mix of faculty and students. The lion’s share of data collection and the work of interviewing the grads were done by 51 graduate student researchers.
The project tracked over 2,700 PhD grads (graduating cohorts from 2004 to 2015 from 60-plus departments). The TRaCE website features analyses of the quantitative data as well as narratives developed from the interviews. The website has a networking capacity: once we build a greater online community, the site will enable grads and students to talk about different career pathways. If a student reads a narrative that speaks to her aspirations, she will be able to reach out to the person whose story it is.
We are working now on TRaCE 2.0. It will be a two-year follow-up project that will aim to gather quantitative and qualitative knowledge about the humanities PhDs we didn’t have the resources to include in the pilot and also to reach out to PhDs in the social sciences, as well as the fine and performing arts. Also in the planning stages is a four-year research initiative, the Human Sciences Doctorate in Canada (HSDC) Project. HSDC will draw on the data and narratives from TRaCE 2.0 and will undertake to answer three questions: what do human sciences PhDs do for the people who earn them, for the institutions that offer them, and for Canadian society?
But before we start TRaCE 2.0 or HSDC, we want to have a clear-eyed look at what we’ve done in the pilot project. What did we do right and wrong? What are the key questions that need to be addressed about methodology? How far do we have to rethink the enabling assumptions of the work? To do this, it will be helpful to organize our reflections into three large categories: communication, methodology, and bias and objectivity. There will naturally be a bit of overlap among the categories.
Communication
Communication is a tripartite area of concern. First is the communication internal to the project. We were in contact, usually by email, with administrators, staff members, faculty and students at the 24 participating universities. Email was a mixed blessing. Sometimes, we “over-communicated” with colleagues because we had different head office tasks to complete. We’ll coordinate communication more carefully so as not to waste the time of very busy people at the participating universities.
Going forward, we’ll aim to improve communication among all the participants in the project by developing a clear sign-on process, a more rigorous training program, and a dedicated Facebook page (so that all the participants in the project can talk to each other). We’ll ask each university to nominate the students who will be taking part in the work and also designate the faculty advisors in each participating department.
The second communication question has to do with how we connect with the grads. The first stage of the TRaCE work is data gathering. We do that from public, web-based sources. But the second stage of work, interviewing the grads, requires informed consent. That means, at the start of Stage 2, each grad we’d tracked in Stage 1 got an email invitation as well as a consent form. The emails came from the student researchers, so at least the sender address looked familiar. About 10 percent of the grads agreed to be interviewed. That is not a negligible number, but we think we can do better.
No doubt it will continue to be challenging to recruit interviewees. We encouraged person-to-person communication – grads telling other grads about the project and faculty members talking to former students. The student researchers themselves are the best ones to help get the word out. In fact, owing to word of mouth from the student researchers, we have already had people reaching out to us and asking if they can tell their stories on the site.
Another way to address this challenge is to develop a larger presence for TRaCE in social media and mass media. If grads have seen something about TRaCE on Facebook, read about the project in University Affairs, the Globe and Mail or La Presse, or heard about it on CBC Radio/Radio-Canada, they will be less likely to hit the delete button when that invitation email lands in their in-box.
We’re talking here, by the way, about the third communication area. If we’re able to raise the profile of TRaCE by word of mouth, social media and mass media, not only will more grads agree to be interviewed, but people inside and outside the academy will be interested enough take a look at the TRaCE website.
When they do visit the website, they will see the different kinds of work that humanities PhDs do and how they are contributing to Canadian society. There are PhDs who struggle to find their way, and many who try out several different career pathways, but there are no failures. That category used to be used to describe PhDs who didn’t get tenure-track jobs. Even the non-TT PhDs thought of themselves that way. The TRaCE website is showing how many different pathways there are by which PhDs are able to achieve success and how many different kinds of success there are.
Methodology
The second area of concern, methodology, includes preparing student researchers for the work, providing day-to-day support for them, and ensuring that faculty advisors are prepared to lend a hand when they are called on.
How to do the actual work of the project is where we had to learn the most, and learn very quickly, over the past year. Especially in Stage 1, where we were gathering the basic statistical data, we were very fortunate to have the help of colleagues at the Higher Education Quality Council of Ontario and Lilli Research Group.
We did develop all the pieces and all the procedures needed to accomplish the goals of the project. We didn’t do it perfectly; there is still plenty of room for improvement. With a September 2017 start date for TRaCE 2.0 in view, we’ll get to work revising the data collection and interviewing templates. We want them to be easier to use for the student researchers and also capable of providing sound statistical information and better narrative accounts of the grads’ histories.
Bias and objectivity
The last major area of concern has to do with three kinds of bias built into the sources of information. These biases are in part effects of the university system, of the society in which the university operates, and of traditional academic culture.
Not surprisingly, the web, which is where we gather the basic data, is very far from an even-handed source of information. Since universities have (for very good reasons) invested heavily in their own web presence, grads with tenure-track jobs pop up as soon as they are googled. People who have moved into other established fields in business, government or media are also easy to find. But it is much harder to find reliable career information about many others. Adjunct professors tend to appear on multiple academic websites with very little indication of where they were first and where most recently. The professional pathways of many others, especially grads who are developing hybrid careers (like the wonderful PhD who is a creative writer, cheesemonger and small business owner) are far harder to track on the web.
Since we’re working with only the names of the grads, we have not yet been able to produce anything like a full account of PhD outcomes for people of colour, people from different ethnic and religious backgrounds, differently-abled people, and/or members of the LGBTQ community. We aim to address this kind of bias more effectively in TRaCE 2.0. One way forward is to feature many more narratives about non-traditional grads. Another is to seek to recruit a more diverse group of student researchers. Diversity in the TRaCE team will enable more sensitive data analysis and interviewing.
What all this means is that our data on the grads – we succeeded in getting broad occupational information on approximately 82 percent of them – are skewed toward those grads who have secured steady employment in fields that have a strong web presence. Our information doesn’t fully account for grads in new, less secure, less well-established, or hybrid careers.
The third kind of bias is a reflection of traditional academic culture. That we are affected by this bias is richly ironic since TRaCE exists precisely in order to open up academic culture to reassessment and reformation. The student researchers found that grads with tenure-track jobs were more likely to consent to be interviewed and more likely to want their stories on the website. We welcome the TT grads warmly and we tell their stories with alacrity. But we want also to encourage grads outside the university system – industrious, creative people who are doing valuable, non-traditional kinds of work – to join us in greater numbers and work with us to tell their stories.
As we move forward, we will continue to expand what is known about the work PhD grads do and will continue to build community across the boundary between the university and the world outside the university. We’ll undertake to show that the knowledge and skills of PhD grads are of value not only within the academy, but also critical for the economic, cultural and political advancement of Canadian society. We will work hard on both TRaCE 2.0 and the HSDC Project to suggest how the institutions that create and support PhDs can better adapt to a changing world.
Eliza Bateman, a PhD candidate in law, is the TRaCE liaison officer; Catherine Nygren, a PhD candidate in English, is the TRaCE writer/editor; Paul Yachnin, Tomlinson Professor of Shakespeare Studies, is project director. The authors, all at McGill University, thank Sheetal Lodhia (TRaCE project manager) and Nichole Austin (a PhD candidate in epidemiology and TRaCE quantitative analyst) for their support and good counsel.