The Importance of Questionnaire Design

Planning a survey requires many steps and decisions along the way: “How many people do I need to survey? How am I going to distribute the survey?” And, while people often figure out what questions they want to ask, many overlook the importance of expert, unbiased questionnaire design.

When you want to ask questions with the purpose of reliably representing a large group of people and even making high-stakes decisions based on the answers, it’s important that your questions avoid bias and are not open to personal interpretation. More importantly, they need to gather the information you think they do. In short, ask clear questions that gather straightforward answers instead of riddles that require mental gymnastics.

Questionnaire development is more of a science than an art, and reliance on people with research experience can make the difference between a lackluster or downright disastrous effort and one that turns you into a hero. Here are just a few examples of how attention to the questionnaire can impact the overall project.

Ready to go?

Recently a customer provided us with a ready-to-go survey, which we used as a starting point for the final version. Of the 15 questions in the survey we received, the majority required edits for respondent usability or for clarity. Data collected from the ready-to-go survey would have been misleading—effectively meaning nothing. Furthermore, if the survey sponsors didn’t realize how unreliable the data was, they would have been better off making decisions by flipping a coin. At least then they would have known the gamble they were taking, instead of erroneously relying on baseless data to help inform their decision.

No one likes to do the math!

Since you’re asking your respondent to do you a favor, it’s important that the questions you ask are easy to understand, and your instructions are easy to follow. In a past survey, respondents were asked to provide percentages for 11 categories, which, when added together would equal 100%. It’s no surprise that almost 1 in 5 respondents skipped that question. Missing out on the answers of that many people is wasted opportunity.

You also want to make it easy for your respondents to understand and carry out your instructions. Since they can save space and group similar questions together, a table or matrix format is great to use in questionnaire design. Unfortunately, this format may not always contribute to accurate data collection and they may not display well in an online survey taken on a small screen. For one of our survey projects, almost a quarter of respondents simply didn’t comply with the instructions when presented with a matrix question. On the other hand, when each question was asked separately in another survey, the non-response was much lower.

During the early planning of your survey, consider how the questionnaire will be developed and keep in mind the importance of strategic question writing. Furthermore, if you feel you have all the other details figured out and just need help with questionnaire development a survey research firm will be able to help.