We are often asked how many interviews are necessary to run a credible poll. Part of the answer is technical – you need a minimum number for the results to be statistically robust. In the past most consumer surveys were based on a sample size of 1,000. But the other part of the answer is practical. Surveys need to be seen to be credible by journalists for them to be successful.
Is 1,000 interviews enough?
Some journalists have got the impression that only 2,000 interviews is enough for a national poll. On this they are only half right. There are times when 2,000 interviews are very useful, or essential. For example, being able to cut results by region and demographics (i.e. 18-30 year olds in London versus 18-30 year olds in Manchester). Studies designed to produce audience segmentation analysis also often benefits from a larger sample size, because it allows more detailed analysis of audience segments.
But for general consumer polls, which are not designed to focus on particular cities, or sub demographics, it is not essential to have a sample of 2,000.
The truth is size doesn’t automatically mean quality. Here’s why.
A survey of 2,000 interviews which is not nationally representative is no better (and often less robust) than 1,000 interviews with a sample which is well structured and properly representative.
Why? Well if you have 2,000 completions from an online survey, but the number of men, age groups or regions are not representative of the UK population as a whole, then the sample will not be accurate. Some polling companies do quick surveys based on 2,000 people. The main reason they are doing this is to ensure there are enough people to represent age, gender and region. If the sample is not controlled to be representative, it is not unusual to have 60% or 70%of the sample consisting of women rather than 52% which is the true % of women in the UK population. Similarly, the sample may well deliver too few young men.
In reality, a smaller sample of 1,000 people, but properly controlled to mirror the age, gender and region profiles of the UK population as a whole, will be more representative, accurate, and possibly cheaper to do. So why do 2,000 interviews?
The other issue, of course, is panel quality. Its not the size, but the quality that matters! Read our article on why panel quality matters.
Unless it is essential to drill into demographics by other questions or sub groups, 2,000 interviews does not need to be the minimum size survey. If in doubt about sample size, please ask the advice of your polling company. They should run through the options and let you know how the sample will be structured.