Customer Success

How to Improve Your Survey Design

February 14, 2011

When companies decide to devise a new marketing strategy, redesign a sales process, introduce a new product to the market, or segment their current market, they need primary research that will reduce the risk of their decisions and validate their hypotheses.

Of course, that’s only true if the information culled from that research is actually valid.

One of the most important components of all primary research is developing a questionnaire or designing a survey that asks the right people the right questions with the right flow. Following that process will lead to the acquisition of usable and reliable data, providing companies with a framework that they can use to formulate well-educated strategies.

So, I suggest introspectively looking at the questionnaires or survey designs that you’re currently using. Are they accomplishing that task?

Here are a few questionnaire design tips for the product, marketing and research teams of expansion stage and start-up software companies:

What’s the Objective?

It’s very important to keep in mind the objective of your research and what you’re aiming to achieve throughout the survey. Doing that will help you focus on the questions that really matter and avoid wasting time on the ones you may never use in your analysis. So, keep it short and relevant.

If you don’t have collective agreement on what your ultimate objective is, Nobles Research provides a nice guide that suggests ways to define it and execute around it. Here are a few of the article’s suggestions:

  • Determine the information that is critical and start with the end in mind. Ask yourself: What do I want to have learned when the research is complete?”
  • Review objectives with senior management to ensure that they agree with and endorse them. Once they’re approved, assign one person to “own” the research and be the person for the project.
  • Before conducting the survey or questionnaire, go over your objectives one last time to make sure no adjustments need to be made.

Make it Clear

You don’t want survey questions to be confusing because they’ll likely yield poor data. They should be clear and to the point. For instance, if you ask, “Which software do you use to manage your customer relationships?”, the respondent may consider the question as more of a personal preference query rather than one addressed to the entire firm or team.

If the respondent recently changed to a competitor’s product, then “when” dimension questioning may be more relevant. So, in this case, a better question might be: “Which software has your firm used for customer relationship management in the past six months? If you used more than one brand, please list each of them.”

Regardless, Purdue University’s research writing lab argues that questions can be written or directed in a lot of different ways, but they should never be biased, overly wordy, or assumptive.

Don’t be Vague

Open ended questions invite the respondent to interpret the survey and provide sometimes vague or indirect answers. It’s best to keep most questions multiple choice or yes/no. These types of queries will yield definitive replies.

Open-ended questions are effective in some circumstances. For instance, if your research is qualitative or exploratory in nature, it’s not a bad idea to allow the respondent a chance to write a free form answer.

Use a Scale

Some questions can be quantified by having respondents rank their answers on a scale (for example: from 1 to 10). For questions related to needs, wants, and attitudes, it’s good to utilize a scale because it will make any future analysis easier. It will also allow you to calculate the average of all needs and compare each of the needs to the average to derive the most important ones.

A sample question to consider: “How important is ease-of-use of product X to you on a scale of 1 to 5, where 1 is “not at all important”, 2 is “not very important”, 3 is “somewhat important”, 4 is “very important” and 5 is “extremely important”? You can use a variation of this scale depending on the nature of your research and its needs.

Stick to a Sequence

The order of your questions should follow a logical sequence. Some classification questions, such as those centered on demographics, should always be placed in the beginning or the end. Also, you should structure the outline of your questions so that each one naturally leads to the next. Keep in mind that the order can affect the response. To correct that, distribute half the questions in one order and the other half in another order.

Qualtrics, a survey design and data analysis company, suggests the “Funnel Technique,” where questioning sequence starts with easier, broader queries that act as a way to warm the respondent up. The toughest questions are then placed in the middle, with a set of easier questions to close out the survey. The idea is to keep the respondent engaged throughout the survey.

Test and Revise

Role playing is one of the most important parts of the testing process. Check the questionnaire length by conducting role-playing interviews with your colleagues, asking them about the wording and order of the questions. Adjust and iterate the questionnaire based on their suggestions.

You should always do multiple role plays to simulate different types of respondents and situations. Certain questions may be specific to certain types of respondents, so don’t ask everyone the same questions.

It’s important that your questionnaire and survey design remain fluid and in tune with your market, its customers, and anyone else that the research is designed to pinpoint.

So, make sure to reflect on your survey design and ensure the relevancy of your survey process. Otherwise, the data you collect may not be as reliable as you think. If that’s the case, using skewed or faulty data could ultimately have a negative impact on the strategic decisions you make.

Co-Founder

Faria Rahman is the Co-Founder of <a href="https://www.treemarc.com/">Treemarc</a> which, uses machine learning to make it easy for businesses to order custom packaging and product nesting in a few minutes. Previously, she was a Senior Associate at Northbridge Financial Corporation, a leading commercial property and casualty insurance management company offering a wide range of innovative solutions to Canadian businesses. Faria also worked at OpenView from 2010 to 2011 where she was part of the Market Research team.