When the only tool you have is a hammer…

One of my favorite sayings is from Abraham Maslow:  “When the only tool you have is a hammer, every problem starts to look like a nail.” In other words, people tend to approach problems in a manner dictated by their own strong points or expertise.

As someone who was once an analyst for a political pollster, I know how powerful the temptation is to try to explain any given thing by taking a survey on it (a temptation shared, I’m sure, by journalists), and there is indeed a broad range of questions that can be answered with the right poll, to a considerable degree of accuracy.  However, with due respect to the polling staff at Stanford U. and AP/Yahoo! News who took this one, the question, “Are people voting the way they are out of racism?” is not one of them.

The strength and pervasiveness of any given person’s or group’s tendency toward racism is unquantifiable, and therefore resistant to statistical analysis.  Directly asking complete strangers about their own attitudes toward race, given how dicey a subject it is, is unreliable.  Doing it in an online poll, the samples of which are by definition self-selecting and thereby not random, makes the findings even more unreliable.  On top of all that, making it a sample of “adults,” the single loosest acceptable methodology in polling (as opposed to “registered voters” or “likely voters”) makes it more unreliable still.  Using a pattern of questioning no one else uses, robbing you of any objective mode of comparison, and you might as well not bother.

Not that any of this ever stopped the AP, the analysts for which looked at the results of this online survey, polling self-selected adults, using techniques that have undergone little if any peer review, about matters that defy efforts to pin them down statistically, and concluded that white racism may well cost Obama the election.

Click on the link above to read the AP’s release.  They have also uploaded a PDF file containing the complete topline of responses, for what little those are worth.  Read also the details of their wackadoo methodology.

Someone please inform the AP that no matter how vast or comprehensive a poll you may think you’re taking, not everything can be measured by a poll.

Advertisements

2 thoughts on “When the only tool you have is a hammer…

  1. Pingback: Where does AP get its poll samples?? « WitSnapper | The Blog

  2. Pingback: Could the polls be wrong? It’s happened, but… « WitSnapper | The Blog

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s