As a consumer behaviour expert I’m very wary of consumer research and that includes opinion polls. I can see that a huge amount of human behaviour is unconsciously driven and just because we can come up with reasons for our past or intended actions doesn’t mean these rational justifications are accurate.

David W. Moore spent many years working for the polling organisation Gallup as a senior editor. His analysis of opinion polls provides many reasons why we should be concerned about quantitative consumer research in general.

Beyond the general issue of unrepresentative samples he reveals how different approaches to polling produce dramatically different results.

• The complex questions prior to the reported opinion prime respondents to answer in a particular way. One poll asking about support for oil drilling in the Alaska’s wildlife refuge found the public opposed to it by a margin of 17 percentage points. Another poll conducted within a month of the first found the public in favour of it by exactly the same margin (both polls corresponded with the interests of the people commissioning them).

  The difference? The poll that found more people in favour of drilling preceded that question with thirteen about the cost of oil and the country’s dependence on foreign suppliers. The poll that found more people against asked only the question on drilling in that region of Alaska.

• Another issue arises from the fact that people will give an opinion if they’re asked, but that all opinions are not the result of equal thought or awareness of the issue concerned. When people were asked if they thought the US government should spend money on an anti-missile shield the results appeared fairly conclusive: 64% thought the country should (only 6% were unsure).

  When the pollsters simply added the ambivalent option in the question “… or are you unsure?” the level of uncertainty leaped from 6% to 33%! When they drilled down marginally below the surface and asked whether respondents would be upset if the government took the opposite course of action from their preferred route 59% of people either didn’t have an opinion or didn’t mind if the government did something different: a far less compelling picture.

• When polls ask about issues in an abstract way they also change the response: for example, it’s easy to be in favour of something when you haven’t considered the cost.
• It’s easy to assume a level of understanding that doesn’t exist (and regrettably people’s ignorance doesn’t stop them giving an opinion when they’re asked). And it’s also easy to get swept along (especially if you’re in the media) with an inference about a question that ignores the specifics of the question itself. I certainly felt sure that I’d heard that most American’s believed in creationism.

  In fact one poll found that 68% of American’s supported the teaching of creationism along with evolution in public schools. This in itself would be a scary enough statistic for anyone familiar with Darwin’s work, were it not the fact that no one bothered to explain to respondents (or ask if they understood) what creationism is.

  Perhaps I’m being overly generous but it’s certainly possible that people were basically saying, “Yes, teach lots of stuff in schools. Now have we finished with the tedious questions and can I go?”

My own view is that even the perfect poll (if there were such a thing) is deeply flawed. It only takes one environmental factor to change and people will shift their behaviour (or opinion).

How much faith would you put in consumer opinion to shape your commercial decisions I wonder? And how happy are you to think that your country’s politicians may be using them to inform their policies?

Source: The Opinion Makers: An Insider Exposes the Truth Behind the Polls David W. Moore. Beacon(2008) 

Image courtesy: Collin Anderson