It seems that a new research methodology comes to market a few times a year. And every year, researchers claim that the new methodology the only methodology that we will ever need. That is simply not true. Marketing problems are so variable, and respondents differ so greatly, there cannot possibly be a one-size-fits all methodology.

Choose your marketing research methodology based on the specific requirements of each survey project, the available sampling frame, the timeline, and the budget. Additionally, to choose the optimal methodology for your project, we propose evaluating each method on four categories of factors:

1. Task Factors are first. These are the specific requirements of your project that different methodologies can meet to varying degrees. Think about the diversity of questions the methodology can accommodate. Not all questions can be asked effectively in all settings, and not all tools are flexible enough to ask all types of questions. Some methodologies are better or worse for different questions (think grid questions in mobile surveys). Consider whether you need respondents to evaluate stimuli and what form those stimuli might take. It is very difficult to present graphic, text, or video stimuli to phone interview respondents, which would be a significant drawback to choosing that methodology. The quantity of data needed is also a concern (especially when combined with budgetary limitations). Sometimes, conducting as many in-home interviews as you need for a given project is not feasible. Finally, response rate, a concern in all methodologies, should be evaluated against your intended analysis.
2. Situational Factors are second. Does the chosen methodology give you enough control over the data collection environment to accommodate your project? Doing an in-home interview with a busy parent with three kids running around may not be optimal for her to focus on the task. Additionally, do you have sufficient control of your field data-collection force? It is critical that each survey is administered consistently to control bias. You should also consider the potential for interviewers to inject their own bias into data collection. Finally, which methodology allows you to bring the project in on time and on budget? Social media surveys are notoriously fast and inexpensive, but are these the right respondents and question formats for your project?
3. Respondent Factors are third. Do the respondents feel comfortable with the methodology? Do the respondents believe that their data will remain private and anonymous, or do they perceive a threat of their identity being compromised? Is the methodology likely to elicit socially desirable responses? That is, will respondents give the responses they believe to be more socially or politically acceptable than their “true” answer? How comfortable will the respondents be in giving honest responses to questions seeking personal or sensitive information? Low-incidence populations are another consideration. Suppose you are trying to sample unique or rare respondents or a small geographic area. In that case, you may be limited in the methodology you can choose by the available sampling frame. Finally, consider how much control for respondents is appropriate for your survey. You might like to give respondents a very long data-collection period to accommodate their busy schedule, but your boss wanted the answers yesterday.
4. Finally, the Environmental Factors. If the global pandemic has taught us anything, it’s that everything can change. What’s going on in the world in general? Will that impact your research or potential respondents in any way? If so, which methodology should we choose to meet the research objectives in some other way? For example, consider economic upheaval, such as we have seen with recent inflation and the potential for a recession. Then, on a micro-level, think about the seasonality of your business and holidays. Trying to survey during the summer is more demanding than other times since summer break, travel comes into play, and response rates are lower. The same thing happened with holidays. Respondents have other things on their minds than taking the time to complete a survey. You still need to make the business decisions, so not getting the information is not an option. Consider what questions you are asking, whether the environment might skew or bias respondents’ answers, and find ways to adjust for the difference.

There is no perfect all-around methodology for every project. You must balance the good and the bad to optimize your survey research. There may be times when you may need to shore up your methodological choice to overcome its weaknesses for your project. Mobile optimization, incentives, reminders, and slider scales are all tools that researchers use to optimize respondent participation. The important thing is to understand how your methodology works, how it might impact your project overall and in data collection in particular, and how those impacts can affect your analysis and reporting.

Contact Symmetric to talk about overcoming these research challenges! CLICK HERE.