Many brand trackers yield poor results because the sample is poorly defined. Yet, even with a good sample, results can fail to tell an interesting story if you don’t know what to do with the data.
By Rich Shank, Sr. Principal & VP of Innovation
Brand tracking is a market research mainstay where it can be easy to be either led astray or produce boring results. This whitepaper touches on two subjects worth considering when trying to tell accurate and interesting data-driven brand stories:
- Sample planning
- Drawing meaningful brand comparisons
Technomic developed its Ignite Consumer program on the belief that a lot of the custom brand-tracking studies had design flaws that started with the type of sample from which they were drawn. A poorly drawn sample will introduce sampling error, which makes accurate comparisons hard, if not impossible. With poor sampling, no one can know whether the interesting comparisons you are trying to make are meaningful.
So, what constitutes a good sample?
The best practice is fully randomized sampling. This is a sample where every member of the population has an equal chance of selection into the sample. Yet, the time and effort it takes to collect this type of sample is far too slow and far too expensive to make sense for most market research projects. In its place, the next best alternative is to use proportional sampling, which is simply making sure your sample’s demographics match the percentages of those same demographics in your sample population.
Where past studies often went wrong is when they set quotas on user groups or specific behaviors. The samples often had to have “X” number of frequent users or “Y” number of respondents who did “Z.” This sampling practice biases the results of a brand tracker and is known as “sampling on the dependent variable.” Another way to put it is that we are predetermining how big our customer base is in the sample design, rather than measuring it accurately with a representative sample of the study population. The result is often an overly rosy picture for a brand by misallocating the opinion/behavior of lighter users. It also blinds the brand strategist from monitoring whether their customer base is shrinking or growing.
…and what makes for a good comparison?
Data are meaningless out of context. The most direct way to put them into context is to draw comparisons. But which ones? This is where the “art” comes into play. Having a framework through which to draw from is helpful. These can come from your experience with the brand and its competitors, they can come from theories of consumer behavior or a hunch someone might have. The art is making your comparisons in ways that target some desired outcome, as well as having a comprehensive understanding of the variables available to you.
Common comparisons include:
- KPI comparisons between brands and over time
- Comparisons between demographics
- Behavioral and attitudinal comparisons between customer groups
Is that really it?
Yes. But it isn’t exactly that simple. Statistical tests have nuances. Theories of consumer behavior can be complex, and our business needs even more so. The key when drawing comparisons is to look for key differences between and within groups and then to place them into the context of your business needs/strategy. Sometimes it’s helpful to have guiding questions that can serve as informal hypotheses. For example, is consumer group A more likely to buy my product than group B? At other times, you may be exploring the data, looking for patterns to emerge that have meaning to your business. This requires making many comparisons, scanning them for major differences and then placing those differences into business context. But sometimes the clues lie outside of our data set, and it often takes some creativity to settle on a valid storyline. To illustrate, this whitepaper includes an overview of how we use our data to tell a nontraditional story that falls outside of the standard, sometimes boring, comparisons listed above.
Steps for Drawing a Good Sample
Types of Brand-Tracking Samples
The first type of sample is one that is purely representative of the market. This sample will only have demographic quotas and these quotas will be proportional to the population you are trying to measure. Everyone would be eligible for this sample, and this is important. The goal is to size the addressable market for your brand, define the customer profile accurately and understand market share/penetration. Without this type of sample, it is impossible to know if our survey results reflect the actual market.
The second sample is the most common in brand-tracking research. These are targeted samples that only draw from specific customer groups or populations. It is highly likely this sample is targeting a brand’s customers or user groups. This sample will provide meaningful assessments of customer behavior and attitudes if it is comprised of the proportion of heavy, medium and light users you have in your customer base. The trouble is, without the first sample, you can’t know what this proportion is. So, brand trackers are most effective when they combine these two samples. Below is an example of a sampling plan that uses this strategy.
In the end, the goal was to collect a sample that could measure the brand’s equity accurately and oversample its customers to provide an in-depth attitudes and usage evaluation for the brand.
How To Tell an Interesting Story
Comparisons are crucial and guiding questions are helpful
Recently, I was asked to write an article about Roark Capital’s acquisition of Subway. The guiding question was why this was a good buy for them. To answer this question, I turned to our Ignite Consumer tracking program to help me understand Subway’s market position a little better.
I knew that the chain had recently launched a new menu after undergoing a refresh. So, I wanted to know whether this had improved the brand’s position in the market.
Knowing that our sampling plan provides a stable base of comparison, I felt pretty good about what I saw. Significant improvements in food quality and visual appeal were matched by increased perceptions of the brand’s reputation and advertising.
Drawing on what I had learned about Subway over the years, it was safe to say the refresh campaign has put the brand in a stronger position than before.
But knowing that Roark is an investment firm, I thought perhaps I needed to go a step further and start making comparisons from outside the brand-tracking data set. This is where the artful side of drawing comparisons takes a big leap. To narrow my focus, I asked myself whether it was possible that this brand improvement could result in a significant financial lift. Here, I drew on Ignite Company estimates of Subway’s annual unit volume (AUV) and the U.S. Federal Reserve’s inflation rate.
To make sense of Subway’s finances, I compared its actual AUVs from 2000 to 2022, to define what would have been expected had the brand just simply grew sales at the rate of inflation.
What I found was intriguing and started to point toward another clue to my original “why Subway” question. I found that the difference between actual and expected revenue (another comparison) started to turn negative in 2013 and declined significantly from there. However, when I compared the start of the refresh campaign to the start of where the decline began to turn positive, I noticed that revenues started to trend positive around the time Subway started ramping up their innovations.
So, while I felt I had to step outside of the brand tracker to fully answer my question, the ability to reliably compare Subway’s customer satisfaction over time helped me link a separate set of financial comparisons to the underlying brand story.
Without these comparisons, I would not have had the opportunity to discover an answer, and here it is—while it’s true that Subway’s brand position has been strengthened over the pasts three years, Roark must have seen the financial potential for a turnaround. By the end of 2022, Subway’s AUVs were 25% lower than would have been expected had its sales merely followed inflation. The difference between those two numbers is approximately $2 billion.
So, if I’m Roark, I’m thinking not only do I have a brand with renewed strength, I also have a significant opportunity to grow the value of the company by merely regaining a portion of that 25% of revenue that had been lost, and that is not small change.
Interested in more content? Learn more about Technomic’s custom advisory services and download free content here.
Subscribe to Our Emails
Unique content, conversation and thought leadership from Technomic is just a click away.