Every now and then I receive an invitation to complete an on-line survey.  They’re normally hopelessly poor tools at understanding real consumer motivations.  They interrogate the wrong part of the respondent’s mind (the conscious mind) and unwittingly influence the part they should be targeting (the unconscious mind).

So today, I thought that, as I plod through the survey, I would include a running commentary of what’s bad, just in case anyone else is thinking of running an on-line survey any time soon. 

The First Few Questions
A few classification questions to see where I live, when I was born, to check that I watch television (the subject of the research) and to check that I’m not involved in marketing, market research, journalism, advertising, public relations or television: arguably I’m involved in all of these but, just like any other respondent, I’ll be ticking the answers that suit my purpose: “None of these”.

Questions
The first proper question asks me what channels I can name, other than the five original terrestrial channels.  This awareness question has evidently been written without any awareness that memory is context-based.  Sitting in front of my computer I may recall one set of channels, but when it comes to watching a programme I will select in a completely different environment and mindset.  Has anyone correlated top of mind channel awareness with anything?  What, you many well ask, will this question prove?  I suspect that a TV channel has been advertising itself and thinks that unprompted awareness is a good measure of their advertising impact; I’m betting they can’t substantiate a link between the two.

I’m now presented with ten channels and asked to say how often I watch it, of if I never watch it, if I’ve heard of it.  My options are “most days”, “at least once a week”, and so on.    I select a programme by the programme’s title, I don’t always know which channel I’m watching.  Fortunately, the television companies have tracking data from boxes that actually monitor what a sample of UK viewers watch: why, you may well ask, are they inviting in this meaningless self-reported data?  Presumably it will drive questions later in the survey, but since I suspect I don’t know which channels I watch when I flick through, this isn’t going to be very accurate.

I’ve just seen I have to do this for sixty programmes; who would volunteer for this kind of pain normally?

Now I’m invited to say “how I feel about each of these channels” ranging from a “channel I love” to “no strong feelings”.  So I will rationalise my feelings about a channel.  What, you may ask, would it say if I watched a channel a lot that I didn’t love?  Or if I love a channel but don’t watch it often?  There is an implicit assumption in the questionnaire – in my case one that is misfounded – that I know what I’m watching and can match my viewing to a channel. 

Oh Dear
I realise this might seem like a writer’s licence, but I have actually been going through and documenting my thoughts step by step.  Having reached this question about “channels that I love”, the on-line survey has broken.  I’ve tried two browsers and several refreshes, but the page doesn’t work.  The table of pre-coded responses and channels has disappeared.  Nothing I click helps.  Going forwards delivers a warning that I need to give my answers, but I can’t because it won’t let me.

So, regrettebly, I can’t get to the heart of the survey.

But already I know that no one should attach much significance to this survey’s results.  At it’s heart is a fundamental assumption that people link their viewing to the channel they are watching.  A lot of TV viewing is done in a very ‘withdrawn’ mental state.  We let a familiar programme wash over us like a sort of brain support system: there’s processing going on, but it requires no effort or involvement from us.

Ask yourself this: what did you watch on television the night before last?  The chances are that, unless it was a significant sporting event or a film that you had actively selected, you won’t remember the titles of the programmes, never mind the channels.

I’ve been commissioned to study viewing behaviour and programme selection behaviour in the past, and neither has very much to do with the conscious mind. 

As always with poor market research, the waste in terms of the money spent on the survey is probably minor in comparison with the cost associated with making the wrong decision and not acquiring

Philip Graves