Improving Sample Sustainability: An Industry Imperative

June 29, 2023 | Jason Thomas

It's been almost 30 years since online data collection revolutionized our industry. Today market research is essentially high tech flooded with new digital tools and lots of money from venture capitalists. Ironically, technology has also affected the bread and butter of our livelihood—respondent participation—for the worse. While technology has made researchers' lives easier, technology has also impacted how we treat our bread and butter.

Between 2000 and 2010, critical metrics for the market research industry peaked. Survey response rates averaged over 25%. There was strong interest in joining online survey panels. There was good data quality. And that was when Internet and smartphone penetration were far lower, not to mention far fewer people on the planet. Given more Internet, more smartphones, and more people around the globe, you might expect a much bigger pool of willing and able respondents. Not so.

Response rates are at an all-time low, and people are not as interested in sharing their opinions and taking surveys as earlier. Overall feasibility, data quality, and filling certain survey quotas have become major issues. Unless we researchers identify root causes and reverse course, unless more people want to share their opinions again, data collection will only get worse.

What Happened to Respondents?

A root cause is neglecting the Golden Rule. That is, researchers and their technology began mistreating respondents. Amid the flood of new and better digital tools, in the desire to ride the digital wave, researchers simply forgot about our bread and butter. We forgot about treating respondents as we'd like to be treated were we in their shoes. After all, why share my honest and thoughtful opinions if you're going to abuse me with long surveys, constant requests, and little, if any, compensation for my time? Goodwill only goes so far. Participation and data quality (i.e., a willingness to offer honest and thoughtful response) have also suffered as a result our negligence.

What Happened to Researchers?

Researchers and clients demanded faster, better, and cheaper in response to business pressures. Sample companies were also squeezed by having to lower both margins and respondent compensation. The result was a domino effect. When margins decreased, there was also less money to recruit and build panels—to bake bread and make butter. And baking bread and making butter are by far a sample provider's biggest expense.

Moreover, researchers placed too much emphasis on survey design and analytics and client UX while neglecting the respondent's survey experience and satisfaction. In fact, about 60% of respondents claim to have had terrible survey experiences ( It's safe to assume that survey length and endless grid and rating scale questions are among the chief offenders.

How to Improve Sample Sustainability

How do we reverse course and grow our dwindling pools of respondents? How do we make samples and sampling sustainable? First, the problem won't be solved by just sample companies, sample aggregators, or researchers and their clients alone. The problem is industry-wide.

Coming up with solutions won't be solved in a few hours, a month, or a year. But here are a few ideas to start with from someone invested in sample and panels:

  • Pay a fair incentive. There is a direct correlation between incentives, research quality, and respondent participation/engagement.
  • Shorten respondents' time in the routing system. Many sample companies and sample exchanges utilize a routing method. This means they send samples to a platform and the platform asks question after question to match the respondent with a survey that fits them. While this is an important concept that has helped the industry grow, it is also a very dangerous tool because there are few rules or guidelines to follow. Some sample companies allow their members to go into routers for 10 to 15 minutes or more just getting question after question, many of them repetitive. Respondents become exhausted and frustrated. We should limit any respondent's time in the router, no more than 3-4 minutes. This will improve a respondent's survey experience, satisfaction, and likely their quality of response.
  • Do unto others when designing surveys. Would you want to take this survey? If not, then do something about it. Try to stick to a maximum length of 10 to 15 minutes. Research is expensive, and you want to squeeze every question you can into the opportunity. Yes, understood. But even more important is the quality of response your instrument yields. After 15 minutes, quality declines significantly. So help yourself and your client by keeping it short and succinct to get the maximum bang for your buck and to enhance a respondent's experience during their engagement.
  • Demand transparency and visibility from research partners. Sample providers should increase sample quality by implementing better procedures to aggressively weed out bad respondents. Sample buyers should consistently ask sample providers (even those used as partners) about their quality procedures and refuse to use those who do not have adequate quality controls. Further, sample buyers should refuse to use providers who will not respond honestly (or clearly) to questions about sample sources, quality practices, and processes.
  • Cast a wider net. Consider methodologies other than online surveys. CATI surveys offer immediate respondent verification and validation by real human interviewers. Mail research might also produce better data quality depending on the sample requirements.

Our goal is to tweak processes and make improvements to stop losing respondents willing to take surveys by treating them better as an industry. As previously mentioned, we are all in this together, and it will take all of us addressing the situation to make the needed change. Without that change, the very existence of the market research industry may be at risk as bad data quality reduces the value of insights in business decision-making.

Do your part for the market research industry through sample sustainability! Contact Symmetric today

The challenge is worse for specialty B2B or niche samples, so check out our tip sheet: