Yes, I know I blogged earlier that I didn’t want to blog on polls, because they’re unreliable this year due to the complete absence of solid and consistent standards, demographic balances, or turnout models from one poll to the other. However, I’m going to blog on polls once again before the election is over, and I’ll be staying true to my word because it’s another blog post on polling unreliability.
OK, I also know it sounds like I’m trashing a field of professional study of which I was a part for some time, but I’m really not; I still believe in the basic soundness of political polling. I simply think this year’s election dynamics are unique in their unpollability. However, there is an area of political polling in which I have never had any faith, at least not since I knew what they were. The area is that of the damnable exit poll.
More wonkery, notes of caution for Obamacolytes, and reasons for optimism for McCainiacs, below the break.
Every two years, the Voter News Service (VNS) stations interviewers with clipboards outside polling places to ask people coming out who they just voted for, along with certain demographic markers. The results are sent back to VNS and compiled in an attempt to get a handle on the running mentality of those who actually show up to vote (which, of course, is a step up from the “registered voter” or “likely voter” models you get from most of the polls you see during the season).
However, there’s a fatal flaw in this methodology: the sample is, at least to some degree, self-selecting, and somewhat resistant to demographic weighting and balancing. Exit polls skew toward those voters eager to talk about their votes with strangers; i.e. who are enthusiastic, possessed of above-average education, and trusting of pollsters and the media. All three of these voter subsets historically lean Democratic, this year especially so. Obama voters this year are enormously eager to talk to exit pollsters; given that 18 of 20 Democratic primary exit polls this year yielded results that overshot Obama’s actual vote share by an average of 7 points, the numbers appear to bear that out.
The damage is compounded by VNS data that leaks during the day, giving the impression of momentum that can’t possibly be confirmed. (An historical aside: on the day of the 2002 congressional elections, the entire VNS apparatus broke down, and no exit poll data was supplied to the media or the public all night. The reporting was actually quite orderly that year; I think only the willfully blind would find themselves incapable of connecting the dots. The congressional exit polls of 2006 were also quite far off in predicting contested races, and the 2004 presidential exit polls overshot Kerry’s voter share by 5.5%, far enough off to prompt a study into what went wrong.)
I don’t usually post material reproduced directly from campaign press releases (this one is from a pep talk sent out today by the McCain-Palin camp), but I’ll make an exception in this case since it confirms what I’ve long suspected and said both on and off this blog:
Here are the key points to keep in mind when the exit poll data starts being leaked:
- Historically, exit polls have tended to overstate the Democratic vote.
- The exit polls are likely to overstate the Obama vote because Obama voters are more likely to participate in the exit poll.
- The exit polls have tended to skew most Democratic in years where there is high turnout and high vote interest like in 1992 and 2004.
- It is not just the national exit poll that skews Democratic, but each of the state exit polls also suffers from the same Democratic leanings.
- The results of the exit polls are also influenced by the demographics of the voters who conduct the exit polls.
After the 2004 election, the National Election Pool completed a study investigating why the exit polls that year showed John Kerry overperforming 5.5 net points better than the actual results showed him to have done. Their conclusion was that the primary reason the exit polls was that Kerry voters and Democrats were more likely to participate in the exit polls.
Over at NRO, Jim Geraghy’s professional mentor, codenamed “Obi-Wan Kenobi”, gives a fascinating roundup of all the reasons why the polling community has put itself way out on a limb this year with its wild assumptions as to party balance and turnout figures among voters. Obi-Wan also touches on the unconsidered effect of Republican backlash against the media for sheltering Obama, degrading Sarah Palin, and tearing down Joe the Plumber. (Definitely click the link to read the whole message from Obi-Wan. More here.)
WitSnapper | The Blog 