Bad surveys = Bad insights
The Business of Food
by Jennifer Barney
Bad surveys = Bad insights
2-min read
You can really learn a lot from surveying consumers. You can also spend a lot of money and not learn anything actionable.
Conducting consumer research has traditionally been the job of consumer insights. These researchers are not marketers – they are the voice of the consumer – they tell marketers what consumers think and how they will behave so marketers can make decisions.
Lately, brands are doing their own consumer insights. Using DIY tools like PickFu, 1Q, Suzy, and others, you can survey consumers fairly inexpensively. What these tools offer are guaranteed respondents (member audiences) and real time results. For an extra fee they offer programming (launching on the tool).
But what they don’t do is understand your objectives or develop the right questions to ask.
I’ve seen this go waaay wrong too many times – brands spending on surveys and coming up empty on insights.
Problems usually stem from:
Not having a research objective at the outset
Using the wrong methodology
Making inferences from insignificant data
Take the following steps to avoid pitfalls:
1. Determine the research objectives –This is the biggest input when determining methodology
Examples of research objectives:
Product naming and packaging
Purchase interest
Which consumer target will most benefit
What need and occasion best fit the product solution
2. Use the proper methodology – A survey may not be the right method! It might be interviews with a few select respondents to understand what even are the questions to ask.
Qualitative
For example, if you want to know what different groups of people think of your product, it’s better not to guess at a bunch of questions and throw them into a survey. It’s better to conduct focus groups. This is called qualitative research and this type of research is verbal and in depth. It allows for open discussion and investigates the reasons behind behaviors and beliefs. Once you’ve identified themes and patterns from a few small groups, you form specific questions to ask a wider audience in a survey.
Quantitative
Surveys are quantitative – they collect numerical data from a number of respondents that can be extrapolated to a larger population.
3. Make it statistically significant – You only need to have passed Intro to Statistics (thanks Dad!) to understand that the larger the sample size the smaller the margin of error. For example, a sample size of 1000 has a margin of error of 3%, a sample size of 100 has a margin of error of 10%, and a sample size of 50 has a margin of error of 14%.
Start with screener questions to bring the proper amount of people into the survey, and think about all the different ways you’ll want to cut the data after the results come in.
What often happens is brands field a survey without predicting the incidence rate of the segment breakouts they care about, only to find there weren’t enough respondents to get good data.
Example:
Plantain chip brand marketing to gluten free people conducts survey to 500 general population respondents
Results:
Grocery shopper decision makers = 70 % (350)
Chip shoppers 90% (315)
Gluten-free chip shoppers = 10% (32) – not statistically significant
To avoid wasting time, energy, and money on statistically insignificant, aka useless results, start with screener questions to bring the proper amount of people into the survey. Then think about all the different ways you’ll want to cut the data after the results come in.
We already have an email list and past purchasers list – can we use them to conduct surveys?
You can absolutely survey your own people! I’ll explain the best way to do this next week.
All my best,
Jennifer
FoodBytes! Pitch consists of private mentorship sessions, one-on-one connections and media opportunities for selected Ag Tech, Food Tech and CPG startups. Check out this year's winners
I'd love to hear from you - get in touch at jennifer@3rdandbroadway.com