We are entering the season of the political poll. Between now and Alberta’s referendum, we will repeatedly be told what Albertans supposedly think, what they fear, how they intend to vote, and, most importantly, what everybody else is apparently thinking.Question every one of them — including the polls telling you exactly what you want to hear.A poll does not actually tell us what Albertans think. It tells us what a selected group of people told a pollster, after being asked particular questions in a particular way, during a particular period, and after those answers were weighted and adjusted according to assumptions made by the polling company.A straw poll is the most obvious example. It is an informal survey with no necessarily representative sample. An X poll asking whether Alberta should become independent might attract 20,000 responses and still tell us little about the electorate. If an independence account asks its followers about independence, we may simply have established that independence supporters support independence. Hold the celebrations.Professional polling is an improvement, but we should stop treating it as scientific scripture. Someone pays for the poll. Someone chooses the sample and writes the questions. Someone decides whether respondents should be adults, eligible voters, or “likely voters.” Someone determines how to weight age, geography, education, and other characteristics. Someone decides how undecided voters are treated. Finally, someone decides which number becomes tomorrow morning’s headline.That is an extraordinary number of human judgments standing between the voter and the headline.The polling profession itself acknowledges the problems. The American Association for Public Opinion Research identifies sampling, declining response rates, turnout models, question wording, question order, and late-deciding voters as factors that can affect accuracy. Weighting is used to correct samples that do not properly resemble the population — but weighting itself requires assumptions about what the electorate actually looks like..Then there is nonresponse bias. People who willingly answer political surveys may differ from those who ignore unknown telephone numbers, delete survey texts, or refuse online questionnaires. Pollsters attempt to compensate mathematically, but mathematics cannot interview people who never answered in the first place.The question itself deserves equal scrutiny. Ask, “Should Alberta separate from Canada?” and you may receive one response. Ask whether Alberta should “become an independent country controlling its own taxes, resources, and laws,” and you may receive another. Mention pensions, economic uncertainty, equalization, Ottawa or greater self-government, and the response can move again.This means wording matters, which is why federal election law requires disclosure of the sponsor, polling organization, sample population, methodology, weighting procedures, and the actual wording of published election-survey questions. Parliament apparently considered those details important enough to legislate. History gives us plenty of reasons for skepticism.Albertans should remember 2012. Published polls throughout that provincial campaign showed Danielle Smith’s Wildrose competitive or leading. Alberta then voted, and Alison Redford’s PCs received nearly 44% against roughly 34% for Wildrose, winning 61 seats to 17. Polling had not merely missed a decimal point. It fundamentally misread the result.Now we have an even fresher example.On September 14, an estimated polling projection circulated for the Calgary-Shaw by-election, putting the UCP at 54% and the NDP at 40%. It was a projection rather than a conventional riding-level scientific poll, and that distinction should be made clear. But consider what happened only hours later. NDP candidate Kyle Campbell received 7,461 votes, or about 46%, while UCP candidate Mike Derry received 6,879, or about 43%. The projection had the UCP comfortably winning. The voters elected the NDP..That is not an argument that polling is worthless. It is an argument against the false precision with which polling and polling-based projections are so often presented — 54–40 looks authoritative. It looks mathematical. It looks settled. It was also wrong.The United States supplies the classic warning. In 1936, Literary Digest surveyed millions of Americans and predicted Republican Alf Landon would defeat Franklin Roosevelt. Roosevelt subsequently won 523 electoral votes to eight. The sample was enormous — the problem was that it was the wrong sample.Modern methods have improved enormously, but the central problem remains: pollsters are trying to construct an electorate before the electorate actually exists. They must estimate who will vote, how undecideds will break, which respondents are representative, and how heavily each response should count. Change those assumptions, and you can change the number.There is another problem we rarely discuss: publication. A poll may contain dozens of findings, but the public usually sees one headline. “YES Support Collapses” gets attention. So does “Independence Surges.” A three-point movement within statistical uncertainty can suddenly become a political earthquake by the time it passes through an editor, television producer, or partisan social-media account. That is where healthy skepticism becomes essential.Between now and Alberta’s referendum, both sides will produce numbers favourable to their cause. Independence supporters should be no less skeptical when a poll shows YES soaring than when another claims it is collapsing. Ask who commissioned it. Who paid? What exactly was asked? Who answered? How many refused? How were responses weighted? How many were undecided? Were respondents adults or probable voters? And was the headline actually supported by the underlying numbers?Most importantly, never allow a poll to substitute for persuasion.For the YES campaign, that lesson should be particularly clear. A favourable poll will not deliver independence, and a hostile poll cannot prevent it. Calgary-Shaw has just shown a useful difference between a predicted electorate and the people who actually arrive and vote.Pollsters estimate. Commentators interpret. Headlines exaggerate. Only voters decide.So when the next poll announces that Albertans have supposedly made up their minds about independence, do not panic and do not celebrate. Examine it. Challenge it. Ask who paid for it and how they arrived at the number.Then put it down and get back to talking to Albertans. Because on referendum day, the pollsters will finally lose control of the sample. There will be no weighting model, no carefully selected panel, no “likely voter” assumption, and no headline writer deciding what the numbers mean.There will only be ballots. And that is the first poll in this entire campaign that every Albertan should believe.
