The Polls Didn’t Fail, The Pollsters Did

There’s going to be what will become the longest autopsy of all time on the stupendous failure of polls to predict what happened last night. I won’t dwell on the minutiae, but I don’t think we had a polling failure.

The data was there. What we experienced was an interpretation failure of enormous proportions. It was one of the biggest confirmation bias errors the media has ever made, combined with the abject horror (by some) and shock (by all) at what we were seeing.

Nate Silver at FiveThirtyEight repeatedly and emphatically warned that the polls contained a heaping helping or two of uncertainty, which meant the outcomes could vary wildly. He was ridiculed for it by many in the press who would rather just parrot the polling results without a rigorous data-based model.

The LA Times tracking poll, which consistently leaned Trump by 4 points, was more right than the rest of the polls. But it was only right because the grab bag of participants happened to stumble into the correct voting and demographic categories.

And this is the issue with polling. There weren’t a lot of “shy Trump voters” in the sense that they weren’t willing to say who they might vote for. But the models were shy in their assumptions about who will turn out to the polls, in what proportion, and what blocs of voters lean which way.

These are powerful confirmation biases that have been proven false:

  • African-Americans would reliably turn out for the Democratic candidate because President Obama implored them to;
  • Hispanics had enough voting power to overturn a largely middle-class, middle-aged, white voter base;
  • Women would vote with their emotional connection, or revulsion to Donald Trump for his history of misogyny;
  • The correlation of a popular sitting president with a relatively good approval rating to the party in power continuing in office;
  • The failure of issues to dominate the political discussion makes the election into a popularity contest.

President Obama is everything we thought he is, and because of that, he exhibited shockingly short coattails. It is his policies: Obamacare, social experimentation in the military, and his failure to deliver on economic promises, which sunk Clinton. She had nothing new to offer except doing more of what Obama did and failed to deliver on the hopey changey future. Because Trump ran as a demagogue, Clinton thought she could simply run as the anti-demagogue and win.

And the poll interpretations supported that view. Except they didn’t. There wasn’t enough correlation of the issues people cared about with the issues Clinton was touting. In other words, Trump found issues that resonated with the electorate while she didn’t. I’d been touching on that for months (here, here), but Trump’s tendency to eat his own feet submerged the effects of these issues beneath the surface of confirmation bias.

Either Trump is a genius by purposely sticking his feet in his mouth as a disinformation campaign so Clinton wouldn’t see the actual trend and believe her own intelligence reports, or it happened regardless due to natural causes. The disinformation hypothesis is not so far-fetched–it’s how the Allies won World War II and had a successful Normandy landing. Certainly, operators like Roger Stone, Newt Gingrich, Paul Manafort, and Steve Bannon knew how to run a disinformation campaign.

But I think it was simply natural causes. The data was there, but the pollsters themselves focused more on their own demographic models, turnout projections, and voter bloc behavior assumptions than issues-based correlations.

Nate Silver kept seeing the disparities in the polls as a sign of groupthink, and kept warning that this uncertainty could drive results in unexpected directions.

Even when the election results rolled in, the pollsters assumptions and confirmation bias kept the election prognosticators from looking at the real data. They kept looking for the thorn in the rosebush, but it wasn’t there.

It took until 2:30 am for the AP to acknowledge the race call. The Washington Post called Pennsylvania for Trump at 1:38 am, which would have put Trump over 270 except they hadn’t called Wisconsin yet. Everyone had the same data, but nobody wanted to make “the call” to put Trump over. Except one: Decision Desk HQ.

I’ve known Brandon Finnigan for a couple of years. He’s fearless in his calls, he knows Pennsylvania’s electorate better than just about anyone in the U.S., and he’s been building a completely independent crowdsourced election reporting mechanism for 4 years.

An hour and a half before anyone else, Finnigan saw the data, saw that Clinton couldn’t win in Pennsylvania or Arizona, and called the race correctly. It wasn’t taking a chance, it was simply seeing through the confirmation bias (Finnigan had previously concluded here on this site it was nearly impossible for Trump to take Pennsylvania, but shrugged and did what the evidence showed).

In future elections, we will all benefit from this election as an object lesson in confirmation bias. Black Swans tend to have that effect, exposing bad assumptions and herd mentality. The data was there. Next time we’ll do better.