Rankled by rankings
Professor Jean-Guy Blais denounces the "trite" and "invalid" rankings of Quebec high schools published by the newsmagazine L'actualité....
Jean-Guy Blais is a man on a mission. He wants to tell whoever will listen that the annual ranking of Quebec high schools, published each fall in the news magazine L'actualité, is a load of hooey. Dr. Blais says that due to his position - he's a specialist in scientific methodology in the education department at Université de Montréal - he feels compelled to speak out. The ranking, he says, is an insult to his profession.
"I can't understand why the media give so much visibility to something so trite and which contains so much invalid information," he says. "It's my scientific duty to dispute this ranking in the same way that a physician should refute bogus studies coming from the healthcare field."
In 2002, Dr. Blais criticized the high school ranking in the editorial pages of the Montreal dailies. But this past fall he went a step further, holding a press conference to release a study he'd conducted, which concluded essentially that the students are the determining factor in a school's ranking, not the school itself.
In an interview with University Affairs, Dr. Blais says the methodology used in the L'actualité ranking is not scientifically valid. It classifies Quebec's high schools from one to 455 based on a simple average, not taking into account margin of error, measurement errors, and the fact that several schools were tied. The best that can be said is that there are 50 schools at the top, 50 at the bottom, and the rest in the middle.
Dr. Blais is also skeptical of the ranking's sponsors: the Montreal Economic Institute and the Fraser Institute. These institutions declare themselves to be independent, he says, but they have a definite free-market political agenda. "These rankings are used to manipulate public opinion."
In his own study, Dr. Blais obtained, through Quebec's Commission on Access to Information, the provincial writing and mathematics test results from 1994 to 2001 for every student in Grades 10 and 11 (secondaire IV and V). He found that the "differences were much more marked between students than between schools." He noted, moreover, that private schools do much better because they choose the best students. There are also economic differences between regions, and this shows up in the students' results. "That's why I underline that it's not the school that's determinant, it's the student."
The problem with rankings is not going to go away any time soon. The Maclean's university ranking remains popular and garners much media comment, and the Globe and Mail recently jumped in with its own university ranking in collaboration with research firm Uthink. There's even, in Quebec, a ranking of hospitals.
"I think these rankings serve mainly to sell papers," says Dr. Blais. "Some people will find them an interesting exercise, but it's a futile exercise. To classify these institutions in a rank, all you need to do is choose some variables. Any institution can find itself in first place."