So, you’ve decided to call some people and ask them a few questions, let’s say about the next election.
It’s not 1936 anymore so you won’t repeat the polling mistake of the Literary Digest assuming that if they call people on the phone they will have a random sample. It was the Great Depression after all, so not everyone could afford a telephone.
For the record, the polls predicted a 57% Landon win; Roosevelt won in a landslide with 60%.
But today everyone owns a phone, so you can’t fall in the same trap, can you?
Well, calling people doesn’t necessarily assume that you will get a random sample. Let’s assume that indeed everyone owns a phone. However, some people own two or three phones. How do you know if you’re not going to call the same person multiple times?
And what about the time of day you decide to make the calls? Some people will not answer their phone in the morning; other will not answer in the evening.
Some will not answer numbers they don’t know, period.
And then not everyone chooses to reply to polls. Someone who chooses to answer a certain type of question is usually interested in that subject, e.g. politics in our example. That already carries a certain bias on its own.
For example, if you ask people about a car brand the ones who decide to answer are most likely interested in cars. But you want answers from the general population, for example asking people if they recall a car ad from your brand.
That means that the sample you get is predisposed in liking cars in general which gives you a distorted image of the effects of your ad. Because, remember, you want your ad to convince most people that your cars are worth buying – not just the diehard fast-and-furious type of fans.
And let’s not even get into more heated topics like political polls. Just try and think if the person who answers the polls is your average Jane / Joe, or a person who survives on a diet of extreme, one-sided, militant political views.
So, next time you see some poll numbers, be sceptical about the sample and how the poll was conducted.
Similarly, if you reckon polling people is a good idea to find out what the populations thinks about your brand or organization, ask yourself the question: is the “population” being polled really the population?
And this goes for all media that you can run polls on: phone, in-person, online.