The Polls are Always Wrong

Words by Tom Davis


In a few weeks, Australia has its first “post-COVID” federal election. With Prime Minister Scott Morrison’s Liberal/National Coalition haemorrhaging votes on women’s issues, integrity, and climate, pandemic and disaster management, the opposition Labor Party has won (or tied) in every published poll since January 2021. 


That’s why Morrison has, for months now, been firmly on campaign footing. Starting in December with slogans (“personal responsibility”, “riding the wave”, “pushing through”) that played up Aussie resilience on his preferred path to “post-COVID”, Morrison then shifted to national security and increasingly bizarre antics looking for some advantage. But the polls didn’t bounce back. Instead, the mismanagement of Omicron and record flooding, and internal party problems, have pushed his government’s already-dire polling numbers into a terminal decline. Nowadays, the Coalition’s pushing the budget, the “arc of autocracy” thing, and Labor’s controversial workplace culture, but there’s no telling how much any of that will matter come election day. 


So, Labor is winning polls, often decisively – one March poll gave them a 16% lead. But commentators are downplaying it. Why? Well, with most published polls against him, Morrison wasn’t supposed to win 2019’s election; but he did win, casting doubt on every poll since. Which is understandable – the polls have a lot of problems. For one, they regularly contradict each other – with most polls last year putting Labor ahead on the primary vote, a Resolve poll had them trailing 7%. Or, in the same week last December, Newspoll put Labor ahead 6% on TPP while Roy Morgan gave them a 13% lead based on a lower primary vote. 


An explainer: Australia has 151 electoral divisions (“seats”) and uses compulsory preferential voting, a system where voters in each seat rank every candidate running for that seat. The candidate with the fewest primary (or first-preference) votes gets eliminated, and their votes are redistributed to each voter’s next preference. This process repeats until two candidates remain – hence “two-party-preferred” (TPP). Now, there are four major regularly-published polls – Newspoll, in NewsCorp’s Australian; Resolve, in Nine’s Age and Sydney Morning Herald; Essential, in Guardian Australia; and the independent Roy Morgan


Polls typically ask respondents who they’ll rank first out of Labor, the Coalition, independents, or minor parties – usually the Greens, One Nation, United Australia and “other”. To estimate TPP, Morgan asks which of the major parties respondents prefer, while Newspoll bases their estimate on the way preferences flowed at the previous election. (2019’s ratios split 60-40 for Labor, with 65% of One Nation and United Australia votes flowing to the Coalition, 80% of Greens and 60% of Independents to Labor). Essential mixes methodologies, using respondent answers unless the answer is “Unsure,” and Resolve doesn’t calculate TPP at all. In most seats, a vote ultimately flows to Labor or the Coalition, so TPP helps indicate who’ll win the election. 


So, if one poll records a much higher TPP than another from a lower primary vote, someone probably screwed up. Polling problems aren’t new – a 2020 Macquarie University study found Australian polls not only fail regularly to predict elections’ percentage results (or even, occasionally, winners) but habitually underestimate Coalition support. One explanation (among many) is “shy Tories” – supporters of controversial candidates who misrepresent their “politically incorrect” views to pollsters, conforming to mainstream opinion for fear of social rejection. Support for controversial candidates therefore goes underreported, the theory goes. 


And sometimes, when anti-government feeling is high, some who will ultimately vote for that government tell pollsters they support another party to express their outrage (as one January KORE poll noted after measuring a 22.7% Coalition primary vote); the Coalition are in government more often, and are, therefore, more frequently subject to such rejection. And other, mundane explanations exist. Poll response rates are extremely low, and a sample of 1500 (out of 17 million) voters, augmented with weighting and quotas to reflect the electorate’s demographic makeup, turns polls into what are essentially educated guesses. The declining use of landline telephones, meanwhile, forces pollsters to rely more heavily on weighting to wring meaning out of increasingly random samples. 


Mindful of all that – polling failure has major implications for democracy. Recent failures have generally favoured illiberal, populist parties and candidates who have (often actively) destabilised democracy. But it goes deeper than that – past polling failures encourage a belief, fuelled by conspiratorial or anti-establishment narratives, that the mainstream media and elites manufacture polls as propaganda to demoralise supporters. Because conspiratorially-minded voters sometimes refuse to be polled altogether, these narratives perpetuate themselves, exacerbating the underreporting problem, which, in turn, gives the conspiracy legitimacy. 


Interestingly, polls are improving – Macquarie’s study finds that, on average, they’ve grown steadily more accurate over the last 30 years. And, just a few weeks ago at South Australia’s election, polls predicted not just the winner, but the numbers too. But facts like these go overlooked when high-profile upsets engender (and, for some, confirm) suspicions that polls are inherently unreliable. “The polls were wrong” has become a catch-cry of politicians positioning themselves in even superficial opposition to “elites” – see Josh Frydenberg, deputy Liberal leader (hardly an outsider) in a government often accused of populist dog-whistling, framing 2019 as part of the populist wave: “Pollsters got it wrong with Brexit, they got it wrong with Trump, and now they’ve got it wrong … in Australia.” 


In short, bad polls are bad for democracy. Upsets shake people’s faith in the system because they make us question our assumptions about the political world – and all the journalists, politicians and “experts” who helped shape those assumptions. That low-trust environment incentivises destabilising behaviour from political entrepreneurs. With trust in government and institutions again eroding, polls must at least be reliable indicators. To their credit, pollsters have refined their methodologies and sought to de-emphasise “horserace” coverage of polling. And the discrepancies aren’t as bad as they look – Newspoll, Morgan and Essential usually fall within each other’s margins of error on primary votes. Resolve, which regularly publishes low Labor and high Coalition numbers, complicates this loose consensus. But most likely this is just a teething problem – they’re only a year old, and sometimes their numbers are just weird (watch One Nation yoyo from 6% to 2% in early polls). So, if the polls (mostly) work again, what’s the problem?


Yeah, the polls still don’t work. Well, they do – better than before actually, now 2019’s problems are being addressed. But 100% accuracy is impossible, so, especially in close elections, sometimes they’ll just be off. Morgan habitually overestimates Labor’s TPP, and they use respondent preferences and poll by phone, which can be less reliable. Essential, meanwhile, uses “TPP+”, which includes “Don’t Know”s (who are typically excluded from other polls), producing results like “47-46-7”. Different methodologies mean results can’t be meaningfully compared without caveats, but when the polls are wrong, who cares about caveats?


The big issue this time is the way TPP is calculated. The assumption that preferencing patterns from one election will apply at the next election only holds true if nothing meaningfully changes between those elections – but 2022 is dramatically different to 2019. Pay attention to Independents and minors, especially United Australia (which polls around 3%). Altogether, Independents, United Australia and “other” minors’ numbers rose from around 9% to 13% since 2019 (though Resolve’s had them as high as 17%), with that 4-ish% bleeding principally from the Coalition. Some, undoubtedly, are swing voters, but most probably fall into two camps – moderates outraged by unaddressed sexual assault and corruption allegations, inadequate climate action, and general mismanagement; or libertarian and conspiratorial supporters of United Australia and similar parties opposed to COVID controls. 


Votes for both groups will probably flow back to the Coalition – moderates pitch themselves as conscience Liberals, an insurgency rather than an invasion (which is why they’re frequently smeared as leftie infiltrators), and their voters are functionally Liberals; libertarians are similar – right-wingers acting out; and conspiracists consume media that, while not exactly pro-Coalition, reserves its greatest ire for Labor and the Greens (see Dictator Dan et al.). But United Australia’s encouraging its voters to rank the sitting member in every seat last, which completely fries any halfway-decent prediction, while voters for New-Age pseudoscience parties (who preferenced Labor in 2019) are now a wild card. 


Point is, Labor’s blowouts are probably illusory (although not completely – on these numbers they still win). And that’s sort of inevitable – the alternative would be to blow up your methodology by throwing a fuzzy 3-ish% the Coalition’s way on a hunch, hardly good practice for a poll that expects public trust. So, outcomes – either, in a situation akin to the 2020 US election, polls pick the winner (Labor), but the gigantic margin doesn’t eventuate; or Labor again wins the polls but loses the election. The latter is less likely if the margin stays wide, but margins rarely do once campaigning starts in earnest. Either way, even a narrow Labor win becomes an upset, confirmation (for some) that polls are, indeed, always wrong, even if they aren’t actually wrong. In our context of conspiratorial populism, declining trust, economic uncertainty, geopolitical tension, and the increasingly visible consequences of climate change, we should all at least understand why they’re wrong. 


Tom Davis

The author Tom Davis

Leave a Response