A bunch of people, both live and over email, have asked me why I don’t blog more on poll results, especially given that I worked in the business for a number of years.
Admittedly, I’m blogging a lot less on poll-related topics than I did, say, a month or more ago, which is a bit counterintuitive in light of the avalanche of poll results that comes in the last couple of months before an approaching presidential Election Day. Since mid-September, pollblogging on WitSnapper has been especially rare, and even then I’ve typically done so only to comment on the misuse or shoddy execution of polls.
There’s a pretty simple reason: taken as a whole, the polls for this race have been entirely unreliable, and my estimation is that it’s because of pollster panic.
In addition, if McCain were to overtake Obama in the last couple of weeks in defiance of the polls, it wouldn’t be the first (or even second) time in modern election history.
Details and further links below the break.
Now, when I say the polls are “unreliable,” that’s not to say that I think McCain actually has a solid lead over Obama which the polls have missed entirely (I’m not going to make some Nixonesque case for the “silent majority”). What I mean by “unreliable” is that, taken as a whole, the results of these polls have virtually nothing in common. Two national polls just released, by Battleground and NBC News, show Obama with advantages of 2 and 10 points, respectively, during similar time periods. An Associated Press poll, just released (no link yet), shows the race effectively tied at 44 to 43.
DJ Drummond has a lengthy but interestingly argued blog post at Wizbang asserting that polls this year, more than any other, are not to be trusted. He assigns a good deal of venality to the polling organs, in fact quite a bit more than I’d venture (saying essentially that polling comes into greater demand when the race looks like a wild, unpredictable “roller coaster ride” and therefore improves a pollster’s bottom line), but he and I are in agreement on one important thing: pollsters this year appear to be fiddling with the number-crunching process far too much, in an effort to predict what a cross-section of this year’s eventual voter pool will look like. It’s an X-factor (or, more accurately, a wide array of X-factors) no one can seem to pin down to the satisfaction of any objective observer.
Samples and demographic weighting for a survey of the progress of a political race are determined primarily by three things: census data, voter rolls, and previous election results. In this election, all three are problematic:
- Available census data, regenerated decenially, is now eight years old. We’ll be ditching it for new data in two years; until then, we’re using precinct, county, state, and national census demographics measured during the Clinton administration.
- Voter rolls are constantly updated, and therefore more current than census data, but difficult to gauge in many states; some states require far less information than others for registration (such as party affiliation, an important demo in any political survey). On top of that, the rash of voter registration fraud uncovered in the news throws voter roll data accuracy into serious doubt, especially in battleground states.
- Basing turnout expectations (among other data) on previous election results invokes the old cautionary military metaphor of “fighting the last war.” This year pollsters will be using turnout numbers from the last presidential election in 2004; does anybody think turnout numbers from demographic to demographic will be comparable in 2008?
Now here’s the thing: I don’t think the problem is that pollsters haven’t taken these complications into account. No, the problem is the opposite: I think they are obsessing over them, especially that last one, to the point where they’re overcompensating with entirely too much “educated guesswork.” Tweaking numbers so that the demographics reflect what the likely voting public will look like is an entirely acceptable practice, when done in moderation. Repeated tweaking, however, has a cumulative effect on the poll’s methodology in which it eventually fails to meet the minimum threshold for scientific randomness. There’s a line dividing “scientific” and “skewed” that can’t be crossed. Since that line depends on nuanced degrees of randomness, it’s a somewhat blurry line, which is why reputable pollsters do only minimal weighting in order to avoid even coming close to that line. That ethic appears to have gone out the window this year, as pollsters have gone crazy trying to account for new conditions on the ground (not reflected in available data) that make this election unique.
I did mention earlier that the polls have been comparably wrong recently. In 2000 and 1980, mid-October polls had Gore and Carter, respectively, enjoying comfortable leads, only to see them disappear (Gateway Pundit has a post on the matter.) I’m skeptical; both of those elections took place under the influence of crushing Clinton and Carter fatigue (again, respectively). This time the fatigue runs in the opposite direction, but I will still be keeping these recent discrepancies in mind.
Another inherent polling problem to keep in mind is the disrepute into which exit polling has fallen in recent elections. Exit polls, taken by volunteers who wait outside polling places and ask people on their way out which candidate they just voted for, have been ridiculously wrong recently, especially since 2000 (in 2002 the entire VNS apparatus crashed, leaving us entirely without exit poll results…it actually made for a much calmer Election Night). In another post by Gateway Pundit, pollster Mark Blumenthal comments that in 18 of 20 Democratic primaries this year, Obama’s eventual voter share was consistently overestimated by exit polls, by an average error of 7 points. Blumenthal assigns the gap to the fact that Obama’s supporters, who are young, highly educated, and very enthusiastic, were more likely to participate in exit polls.
For my own part, I’ll close with an estimation I gave my brother-in-law last weekend when he asked me point-blank whether I think Sen. McCain can still win this election, despite the polls. I told him that while I won’t go so far as to say he will, I definitely think he can still defy the polls and pull it out. I also said that I think the Electoral College vote margin, and definitely the popular margin in the battleground states, will be extraordinarily thin, and in the end, the outcome will resemble 2000 in that the final electoral vote distribution will be decided by election lawyers, not by hopelessly weight-addled polls.