Bot prevention devices are becoming increasingly used to identify automated respondents. One
of the most popular, CAPTCHA (which is actually an acronym that stands for Completely
Automated Public Turing Test To Tell Computers and Humans Apart) can be used in the screener
to eliminate machines programmed to respond to surveys.
All of these methods are effective and can result in removing between 2% and 5% of completed surveys.
A Good Start, But Not Sufficient
The problem becomes that many respondents are onto our methods – and know how to beat them. To
truly improve data quality, we need to take it a step further. We need to add humans to the data
For example, Symmetric Sampling adds a single open-ended question to each survey that asks: “For
quality-control purposes, please tell us the main topic of this survey. Be specific and include details.” We
then have our data quality team review every response and make the judgment to keep or eliminate
that respondent. If the respondent has been taking their time and reading and answering the questions
thoughtfully, they will be able to provide an acceptable open-ended response. However, if they have not
been reading carefully, they cannot give an adequately detailed and specific response. We remove those
respondents, adjust the quotas, and collect better data from engaged respondents.
We have found that this simple approach identifies another 5% - 7% of bad respondents. That’s right –
an additional 5 – 7%, bringing the number of respondents removed for low quality to between 7% and
12% of the total sample.
This approach adds very minimally to the cost and delivery time of the study. We have found that most
researchers find this approach extremely appealing, even when using samples drawn from their
customer base. After all, if more than 7% of your responses are low quality, that could potentially be a
much higher cost for your company in misinformation and bad decisions!
If you are concerned about sample quality impacting your research results - and who isn’t? - consider
using this method. You owe it to yourself, and you might be surprised by the result!
Concerned about data quality? Add the Human Touch! Contact Symmetric today:
Address: 600 Ave H East,
Arlington, TX 76011
Phone: (817) 649-5243
Email: [email protected]
President of Symmetric, a Decision Analyst company