Thursday, November 8, 2012

How polling became credible again -- in the right hands

One of the lessons from this year's presidential campaign is that individual polls really don't matter. The more important lesson is that being able to aggregate the polls into something discernible matters a lot.

There's more to it than just taking all the polls and averaging them together. Some polls are more credible than others. And therein lies the genius of Nate Silver. I followed Nate Silver before following Nate Silver was cool -- starting in the summer of 2008. Now, he's famous and about to become rich, thanks to the huge uptick in sales of his book, "The Signal and the Noise -- Why Most Predictions Fail but some Don't."

Silver's model weights the various polls based on past performance and other factors, and takes into account issues like the latest unemployment figures.

Here's what I find most interesting about Silver's work this year. The electoral map he posted in June is very similar to the electoral map he posted the morning of the election. By September 6, it was identical to what he would post election morning. In each case, he had correctly projected all 50 states (giving some, obviously, a higher likelihood than others). In other words, after spending billions of dollars, the candidates did not change the outcome by a single electoral vote in the final 60 days.

Just as remarkable, while some idiots (Karl Rove, Dick Morris) were suggesting not only a Romney win but a Romney blowout, Silver projected the popular vote outcome within one-tenth of a point (though the final numbers won't be available for several more days). Rove was so clueless that he argued with the experts at Fox News when they called Ohio (well after NBC and CNN had called it). While others were insisting the election was a dead heat going into the last week, Silver had Obama as a 91-to-9 percent favorite to win, all based on a complex model that must be locked in the same vault as the Coca Cola recipe.

When people on the Post Register's message board would refer to this poll or that during the campaign, I would refer them to Silver, mostly because he's non-partisan and his 2008 and 2010 track record was so spectacular. In 2012 he really outdid himself. The stunned people at the Romney "victory" party on election night would have been well-served to have paid attention to Silver. At least they would have been better prepared for the outcome. I cautiously suggested to an acquaintance who had spent the entire summer and fall working on the Romney team that he might want to pay attention to what Silver was saying. I haven't spoken with him since election night, but I hope he took my advice.

There are other important lessons from Silver's work. For example, while Republicans argue that Hurricane Sandy stopped Romney's momentum, Silver pointed out all along that Romney had no momentum after the second debate and was already losing ground to Obama when the hurricane hit. Silver had Obama narrowly winning Florida even when some big polls had it the other way. And, when Gallup came out with polls showing Romney pulling ahead in the popular vote, Silver doused it with buckets of cold water.

Yes, polls matter, in the aggregate and in the hands of geeks who know their numbers.

No comments:

Post a Comment