Barry Kay, a member of the Laurier Institute for the Study of Public Opinion and Policy, or LISPOP, has been doing seat projections for upcoming elections for the past 35 years. But, he warns, “People should understand I do not have a crystal ball. The fact is the model is only as good as the polls it is based on. If the polls are off, it will be off.” And, the bad news is that the polls are getting worse, he says.
Seat projections, as opposed to party popularity, were a novelty when Dr. Kay first started out but have attracted greater interest over the past decade or so. An associate professor of political science at Wilfrid Laurier University, where LISPOP resides, Dr. Kay says his model has been accurate to within four seats per party over the past 15 federal elections.
For provincial elections, however, there have been hiccups: in the 1990 Ontario election, which saw Bob Rae’s NDP win a majority, the LISPOP projection “was not even close.” In the most recent Ontario election, LISPOP projected 49 seats for the Liberals, but the party ended up with 58. The polls, says Dr. Kay, were the problem: “When you put the polls together, it looked like about a 2 to 2.5 percent lead for the Liberals over the Conservatives, but in fact it was closer to seven points, and that was the difference.”
For his projections, Dr. Kay uses what he calls a regional swing model. “When polls come out, I don’t care about the national numbers. I’m only concerned about the regional splits,” he says. The idea is to look at the change from the previous election, region by region, to the aggregate recent polling for those regions, he says, and then apply that swing to each of the ridings in that area. “There is lots of other stuff that goes into it, but that’s the core of it,” he says. Pollster Éric Grenier’s Threehundredeight.com blog, which recently shifted to CBC under the name Poll Tracker, uses a similar methodology.
It is “almost beyond question” that the accuracy of polls is getting worse, says Dr. Kay. “I think anybody involved in the public opinion industry would agree.” As exhibit A, Dr. Kay cites participation rates – the proportion of people contacted who cooperate for the full polling interview. In the 1970s when he started out, the rates hovered around 70 to 75 percent; that has now dropped to no better than one in 10, he says.
The method of polling has also changed, from door-to-door interviews to telephone calls using landlines, and now through online polls and robo-calls, also known as “interactive voice-response” polls. The latter two are particularly problematic because you don’t know who’s answering, and so there is no way of knowing whether certain groups are over- or underrepresented. Telephones are still used for some polling, but many people, especially younger adults, no longer have landlines and cell phone numbers are difficult to access.
“We like to think that technology is getting better in most areas in recent years, and it is,” says Dr. Kay. “But with regard to polling it’s getting worse, and it’s probably only going to get worse still into the future.”
The industry standard is for most polls to be considered accurate within about three percentages points, 19 times out of 20. And yet, two polls taken at exactly the same time can have very different results. Two July polls provide a good illustration of this. One, a Forum Research interactive voice-response poll conducted July 21, showed the NDP at 34 percent, the Liberals at 29 percent, and the Conservatives at 28 percent among decided voters. The other, a Mainstreet Research poll conducted July 20 and 21 and using the same technology, showed the Conservatives at 38 percent among decided voters, versus 27 percent for the NDP and 28 percent for the Liberals. That’s a swing of 10 percent for the Conservatives. “It makes no sense on the face of it,” says Dr. Kay.
Polling firms could improve the accuracy of their polls, he says, but it would cost more to do so. “It’s all about making money. The cheaper you can do your interviewing, the more profit there is for the companies involved.”
The most recent LISPOP seat projections, released on August 13, had the NDP in the lead. As for the final election results, Dr. Kay would venture only one prediction: “It will be a minority government. Nobody’s within a majority territory.”