Universities see benefits with online process for student evaluations

The last day of class used to be reserved for course evaluations – short surveys meant to canvass students about their experiences in the class and with the instructor. But as more universities move to online evaluations, some of them are struggling with what to do about low student-response rates.

Samer Saab, chief executive of eXplorance, a Montreal company that develops and operates course-evaluation software, said more postsecondary institutions are moving to an online process because of the many benefits the system provides. “We’re doubling our business on a year-over-year basis,” he said.

Online forms are less costly because they eliminate printing costs as well as the need for administrative staff to scan and input data. Universities are able to prepare reports for faculty members and senior administrators in days rather than months, said Mr. Saab. This is critical, he adds, because the faster you can produce results, the faster professors and staff can act on the information and the more that students feel their feedback has been heard. “When they feel heard they will want to be part of the process for the long-term,” Mr. Saab said.

However, online evaluations present challenges too, and one of the biggest is lower response rates. The proportion of students completing paper forms is usually higher, at about 70 or 75 percent, because “you have a captive audience,” said Mr. Saab. Students must complete online forms outside the classroom on their own time, so fewer of them tend to do it. Response rates are important, not least because the evaluations are used, among other things, as part of faculty assessment for tenure and promotion.

Universities can take measures to boost participation, he said. They can ask students to complete online evaluations in the classroom, just as they did with paper forms, but using their mobile devices. Another option used effectively by some U.S. schools, he added, is to give students an incentive to complete the forms, such as early access to their final grades.

Mr. Saab advises universities to implement strategies that will lead to a sustained increase in response rates, rather than a one-time spike. These include giving students access to the survey results and, perhaps more importantly, acting on the feedback. “Then students will be very motivated to be a part of it,” he said.

Who has moved their evaluations online

When Dalhousie University moved to an online system in 2012, the average response rate fell to 48 percent in the winter term, down from 60 percent the previous year; the rate fell again to 41.7 percent in the fall of 2013. Dalhousie has launched a marketing campaign to educate students about the importance of evaluations for assessing faculty and informing future course design, said Brad Wuetherick, executive director of the school’s Centre for Learning and Teaching. Dalhousie also integrated course evaluations with its learning management system so that when students log on to the LMS they get a reminder to complete the forms. It plans to create a mobile app so that students can complete evaluations on their smartphones and tablets. Dalhousie administrators hope some of these changes will improve response rates, but even with the lower rates, the university is confident the data is “robust, valid and reliable,” said Mr. Wuetherick.

University of Toronto started online evaluations in 2011 and has been gradually phasing in the process across faculties and departments. Its online evaluation form includes 20 questions, eight of them selected at the institutional level for all divisions. The remainder are chosen by faculties, departments and instructors to reflect their particular priorities. Instructors can select questions from a question bank maintained by the Centre for Teaching Support and Innovation or formulate their own in conjunction with the centre’s staff. Students can access all the data derived from the institution-wide questions and in most cases to the data derived from questions selected by divisions. Responses to the instructor-chosen questions are not available unless the professor chooses to make them so.

“One really important thing we’ve learned about response rates is that you don’t just go online and deal with course evaluations the same way as when they were on paper,” said Carol Rolheiser, director of the teaching support centre and a professor in the department of curriculum, teaching and learning. “There needs to be investment across the entire institution around response rates.”

U of T’s mean response rates for online surveys range from 40 percent in some faculties and departments to about 75 percent. While the mean response rates are lower with the online system, the quality of the data is vastly improved, said Cherie Werhun, course evaluation support officer. Because students have two weeks to complete the evaluations, they tend to provide longer and more thoughtful responses to open-ended questions. She said instructors are pleased with the reports they receive because they are of a higher quality and about 14 pages long, rather than the single page they might have received previously.

The U of T centre for teaching support has looked at departments with high response rates to identify factors that can improve participation. One of the most important is instructor support for the process and the extent to which they use the feedback to improve course development.
At Wilfrid Laurier University, the plan to go online in 2015 was prompted by a move to a new course evaluation form that was longer and more complicated than previously, said Pat Rogers, associate vice-president, teaching and learning. Laurier plans to mount a marketing campaign to educate students about the importance of evaluations and how they are used by the university.

“Teaching is very important at Laurier,” Dr. Rogers said. Course evaluations are just one way of measuring a faculty member’s effectiveness but “it’s an important one because it gives students a voice.” Laurier wants to avoid punitive measures, she said, such as blocking access to final grades to induce students to participate.

Faculty members at some universities have questioned the validity of the data from online evaluations due to low response rates. At least one institution, University of Calgary, reverted to paper surveys in 2008, three years after adopting online forms; participation rates fell to as low as 31 percent in one year.

“We felt we couldn’t live with that,” said Don Best, director of the Office of Institutional Analysis. The figure has rebounded, ranging from 64 to 68 percent, since moving back to paper.