Home » Skills
Developing Surveys
Tuesday, November 26, 2013
If you want to know someone's view on a
particular topic, you go and ask them, right? Getting their answer
directly is much better than second-guessing what their response
might be.
The challenge comes when you want to gauge the opinions of several
groups of people. For example, you may need to assess the mood of
your team or department in response to a new initiative you've
introduced. Or you may want to test out an idea with your customer
base before launching a new product or service.
Whether it's corporate culture issues, or customer satisfaction
research, the best source of feedback and suggestions is usually
the primary stakeholders themselves. We can try to guess what
customers want, or what motivates workers – or we can simply ask
them directly.
That's where surveys come in. Surveys give you the chance to ask a
target group of people a set of questions about a particular
subject, project, or issue. You can then use their views to tailor
your strategy, or review your approach.
However, do bear in mind that effective surveys involve much more
than writing down questions, and asking people to respond.
What, therefore, do you need to know about surveys before you
implement one? And how do you maximize the value you gain from
them?
This article outlines the strategies and techniques that will help
your survey deliver the desired outcome.
Planning the Survey
If you simply sit down and start writing questions, you may risk
missing critical elements of the data you want to capture. Take
time to plan your approach, and decide what you want to achieve.
Consider the following when planning your survey:
Determine your survey goals – What do you want to learn from
your respondents? The clearer you are about what you need to know,
the more focused your questions and answers will be. Do you want
satisfaction ratings, or people's opinions? Are you measuring
attitudes, or predicting future needs?
Identify your target population – Whom do you want to survey?
Respondents could be past customers, potential customers, job
applicants, workers in a particular geographic location, and so
on. Before you define your target audience, assess how their
answers will help you achieve your objectives. Also, make sure
that you have access to your desired respondents. Are you able to contact
the right people to ask your questions?
Be aware of the potential bias
associated with your target population. For example, if you
survey only former workers, or people who have contacted
your customer service department, you're likely to receive a
higher proportion of negative answers. This is because
former staff may have left the company because they were
unhappy, and people tend to contact customer service when
they have a complaint.
Decide your sample size – How many responses
are enough to give you the information and insight you require?
Larger samples usually allow you to reach more reliable
conclusions from the audience's responses. However, you don't want
to survey too many people. Ideally, you need a balance between the
number of responses, and your ability to process and analyze them.
Choose your survey method – How will you deliver the survey? The
time and budget you've got available will often determine how you
carry out the research. Here are some options, and the common
advantages and disadvantages of each:
Person to person – In one-on-one interviews, it's easy to change
or clarify your questions as needed. However, these interviews
take a lot of staff time, so you have to decide if the investment
is worth it. Some companies use telephone interviews to reduce
costs, but this can also be expensive.
Mail – This can be cost effective, and relatively easy to
organize. This method also allows respondents to complete the
surveys at their own convenience. However, there is a tendency for
people to ignore this kind of survey, and there's no opportunity
for you to explain the questions. However, there are ways to
improve response rates, which we discuss below.
Internet – Email and online surveys are growing in popularity,
partly because they're inexpensive to deliver. Email surveys can
be limited in the types of questions you can ask – but you can set
up surveys on websites so that they're interactive, and you can
even program them to react in real time to the responses given.
However, there are disadvantages to Internet surveys. Respondents
must have access to a computer and the Internet. And, you need to
make sure that only the target audience has access to the
survey, and that respondents answer only once. However, there are
various survey tools available – such as Surveymonkey, PollDaddy,
and Google Forms – that aim to make the whole process easier.
Your survey method may create biases. As
we said, online surveys limit your respondents to computer
users with Internet access. One-on-one interviews can be
affected by the time of day – for example, if you're
surveying office staff during work hours, they may be
pressed for time. And interviewing people in one location
may reach only one socioeconomic group, or have another
limiting factor. Consider any potential biases when choosing
your method.
Determine your survey length – How many questions will you ask?
The key here is to be very focused on your goal. If you allow
multiple stakeholders to contribute, you'll quickly end up with a
list of questions that's far too long. It's important to be
ruthless in challenging the value of each question, and to avoid
including the vague 'nice to knows,' or the ones that do not
contribute to your goal. Bear in mind that the survey length may
influence the quality and quantity of responses you receive. When
a survey is too long, people may get bored – and either fail to
complete it, or answer too quickly to save time. If the survey
takes more than five minutes to complete, people may not bother
doing it at all. If the survey is too short, you may not get
enough information to meet your research goals. Also, consider
asking the same question in different ways to evaluate the
reliability and integrity of the responses. KISS (Keep it Simple,
Stupid) is a good guideline when determining your survey length.
Choose the types of questions – Will you use true/false,
multiple choice, ranking, or open answers? The more customized
your responses, the more work it takes to interpret the results.
Simple multiple-choice or limited-choice questions can be scanned
into a computer, and analyzed quickly. However, you may lose some
of the detail by taking this approach, so consider the pros and
cons of each option.
It's important to consider these basic planning elements, and be
clear about your objectives and basic strategy, before you start
writing the survey questions.
Developing the Questions
You can use two basic types of questions in a survey:
Multiple choice – Respondents choose from a series of answers that you provide. This category includes ranking and true/false questions.
Open ended – Respondents answer with a number, or with a written or verbal response.
Here are some useful guidelines for writing questions:
Allow for every alternative – To find out what kind of green
vegetables a person regularly eats, it would be difficult to list
every possibility. Instead, you could list the more common
vegetables, and include a space for 'Other.' Be careful to allow
respondents to include more than one choice. If people eat
broccoli, spinach, and green beans, for example, don't force them
to choose just one.
Allow for Other, None, Don't Know, and Not Applicable responses – Don't force a choice when it may not be relevant. This is also
an opportunity to list only the choices most relevant to your
objective, and then analyze the remaining 'Other' responses
separately.
Ensure that options are mutually exclusive – If questions force
a choice, make sure each option is clear and different. For
example, when asking if workers are full time, part time, or
seasonal, remember that seasonal workers can be full time or part
time.
Make answer choices specific – You ideally want the most
accurate response possible to your survey. For instance, if you
ask people if they've ever listened to country and western music,
they may answer 'yes' – even if they heard only one country song
20 years ago. It's therefore vital to be specific about what you
want to know.
Ask one question at a time – Avoid questions that deal with more
than one issue. For example, consider 'Do you think people should
be allowed to vote early, and have you ever done so?' If people
answer 'no,' does that mean they disagree with early voting, or
they've never voted early?
Limit your choices – Too many options may overwhelm or frustrate
respondents, and they may not read the whole list before making a
choice.
Use an appropriate Likert scale (ordinal scale) – Your response
scale should match the way your choices are worded. A Likert scale
asks respondents to indicate how much they agree with a statement.
Consider, for example, 'I often seek other opinions before making
a decision.' It doesn't make sense to offer the choices Very
Often, Fairly Often, Sometimes, Almost Never, or Never – because
the statement already includes the word 'often.' More appropriate
choices would be Strongly Agree, Agree, Disagree, Strongly
Disagree. An alternative way of doing this is to keep the
original responses, but change the statement to 'I seek other
opinions before making a decision.'
Other response scales:
Excellent, Very Good, Fair, Poor.
Completely Satisfied, Very Satisfied, Somewhat Satisfied, Somewhat Dissatisfied, Very Dissatisfied, Completely Dissatisfied.
Definitely True, True, Don’t Know, False, Definitely False.
None, Very Mild, Mild, Moderate, Severe.
Don't 'lead' the respondent to answer a certain way – Avoid
wording that assumes you'll receive a particular response – and
don't make your preferred answer obvious. Consider the question
'Isn't it best for animals to live in the wild instead of a zoo?'
This will probably lead people to answer 'yes.' A better question
would be 'Where is the best place for animals to live?'
Be aware of cultural issues – People in some cultures are more likely to be agreeable, or aim to create a positive image through the answers they give. Some cultures teach people to give the answer they think is 'correct.' Also, remember to take into account the issue of anonymity. You may want to reassure all respondents that their individual answers will be kept completely confidential.
Ask clear questions – Above all, questions must be clearly understood. Be aware of these potential problems:
Avoid words that may cause certain emotions. For example, words like patriotic, communist, victim, feminist, and insurgent may cause bias in the responses.
Avoid business jargon, acronyms, and technical terms.
Avoid slang and idioms.
Avoid double negatives, such as 'Should we not appeal the decision not to introduce a dress code?'
Avoid ambiguity and vagueness, such as 'We should try to limit access to cigarettes most of the time.'
Designing the Survey Elements
Formatting and laying out the survey is important. Consider these tips to maximize the survey's effectiveness:
Give your survey a title.
Include a short introduction.
Include information about you and your objectives, where possible.
Guarantee anonymity and confidentiality, if possible – or describe how results will be shared, and ask for agreement.
Provide detailed instructions on how to return the survey. Supply pre-addressed, postage-paid envelopes for mailed surveys. Provide contact details where appropriate.
Ask for demographic information (age, education level, income level, and so on) at the end of the survey.
For written responses, provide enough space to answer open-ended questions.
Allow respondents to comment on the survey at the end. Provide space to write responses, or make sure interviewers ask for feedback.
Keep lots of white space (areas with no printed text).
DON'T USE ALL CAPITAL LETTERS – THIS IS HARD TO READ.
Ask pleasant and easy-to-answer questions at the beginning. Save the more sensitive or difficult questions for later in the survey.
For agree/disagree scales, offer the positive answers first.
Mix up the response choices. For example, make sure a 'disagree,' or other negative response to some questions, actually indicates a positive behavior or perspective. This keeps respondents from getting into the habit of always giving an 'agree' answer.
Improving Response Rates
The best-designed survey in the world may not generate the number
of responses you need. This is because people who receive surveys
often don't care enough to complete them. Some of us receive too
many surveys – letters in the mail, phone calls during dinner, and
pop-up screens on our computer when we're working. The smart way
to carry out a survey involves a variety of methods to persuade
people to respond.
Here are three ways that can help you improve your return
rates:
Design the survey well – Follow the design tips above to improve your response rates dramatically. People are more likely
to answer surveys that are well organized, and easy to understand. Keep the survey length reasonable – Don't make your survey too
long. Few people want to answer page after page of questions.
Create a perceived value to the respondent – One of the best
ways to increase responses is to prove that the information
provided will benefit the respondent. For example, if you ask for their input, tell them how their suggestions will be used to impact
their work. When you link responses to a positive outcome, you
help provide clear motivation for them to respond.
The other way to add value is to provide a tangible incentive or
reward. Even a small token of appreciation may help increase your
response rate. And incentives also tend to improve the value of
the responses – you typically get fewer 'Other' and 'Don't Know'
answers.
Finding an incentive that will appeal to your target audience is
key to your success. Here are some of the more common ones:
Tokens – You might include a small, inexpensive item with the
survey, such as a pen or notepad. These are known as pre-survey
incentives.
Coupons – You could provide coupons with the survey package, or
mail them out after a completed survey is received. Typically,
coupons are for the next purchase from your company. However, you
can send coupons for anything you think the respondent might want.
Sweepstakes – Enter all completed surveys into a draw for a
major prize. You can award trips, free merchandise, or even cash.
Other post-survey tangibles – These may include tickets to
shows, dinner certificates, spa certificates, and so on.
Refunds or rebates – For customer satisfaction surveys, you may
offer the chance to win back the purchase price of the product the
customer has just bought.
Remember to issue reminders after sending out the initial survey.
A postcard that arrives a week after the survey may be enough to
motivate someone to complete it. Reminders may not be as effective
as rewards, but they can significantly improve response rates.
Key Points
Surveys are a great way to discover what people think. If you need to know something, asking is more efficient than guessing – but you must ask the right questions in the right way. To develop an effective survey, you should be aware of many factors that affect your design and questions. Have a clear objective, then determine your target respondents, and make decisions about design and content. After your survey is planned, you're ready to develop the questions. This can be an 'art' – and there are many things to consider, including the types of questions, and the exact wording.
A well-designed survey will help improve response rates. But, to maximize responses, you could consider offering incentives. These can range from small tokens to prize sweepstakes. You'll probably invest a lot of time and effort in your survey, so plan carefully to get the most from your responses.
Tags:
Communication, Communication Skills, Skills
particular topic, you go and ask them, right? Getting their answer
directly is much better than second-guessing what their response
might be.
The challenge comes when you want to gauge the opinions of several
groups of people. For example, you may need to assess the mood of
your team or department in response to a new initiative you've
introduced. Or you may want to test out an idea with your customer
base before launching a new product or service.
Whether it's corporate culture issues, or customer satisfaction
research, the best source of feedback and suggestions is usually
the primary stakeholders themselves. We can try to guess what
customers want, or what motivates workers – or we can simply ask
them directly.
That's where surveys come in. Surveys give you the chance to ask a
target group of people a set of questions about a particular
subject, project, or issue. You can then use their views to tailor
your strategy, or review your approach.
However, do bear in mind that effective surveys involve much more
than writing down questions, and asking people to respond.
What, therefore, do you need to know about surveys before you
implement one? And how do you maximize the value you gain from
them?
This article outlines the strategies and techniques that will help
your survey deliver the desired outcome.
Planning the Survey
If you simply sit down and start writing questions, you may risk
missing critical elements of the data you want to capture. Take
time to plan your approach, and decide what you want to achieve.
Consider the following when planning your survey:
Determine your survey goals – What do you want to learn from
your respondents? The clearer you are about what you need to know,
the more focused your questions and answers will be. Do you want
satisfaction ratings, or people's opinions? Are you measuring
attitudes, or predicting future needs?
Identify your target population – Whom do you want to survey?
Respondents could be past customers, potential customers, job
applicants, workers in a particular geographic location, and so
on. Before you define your target audience, assess how their
answers will help you achieve your objectives. Also, make sure
that you have access to your desired respondents. Are you able to contact
the right people to ask your questions?
Be aware of the potential bias
associated with your target population. For example, if you
survey only former workers, or people who have contacted
your customer service department, you're likely to receive a
higher proportion of negative answers. This is because
former staff may have left the company because they were
unhappy, and people tend to contact customer service when
they have a complaint.
Decide your sample size – How many responses
are enough to give you the information and insight you require?
Larger samples usually allow you to reach more reliable
conclusions from the audience's responses. However, you don't want
to survey too many people. Ideally, you need a balance between the
number of responses, and your ability to process and analyze them.
Choose your survey method – How will you deliver the survey? The
time and budget you've got available will often determine how you
carry out the research. Here are some options, and the common
advantages and disadvantages of each:
Person to person – In one-on-one interviews, it's easy to change
or clarify your questions as needed. However, these interviews
take a lot of staff time, so you have to decide if the investment
is worth it. Some companies use telephone interviews to reduce
costs, but this can also be expensive.
Mail – This can be cost effective, and relatively easy to
organize. This method also allows respondents to complete the
surveys at their own convenience. However, there is a tendency for
people to ignore this kind of survey, and there's no opportunity
for you to explain the questions. However, there are ways to
improve response rates, which we discuss below.
Internet – Email and online surveys are growing in popularity,
partly because they're inexpensive to deliver. Email surveys can
be limited in the types of questions you can ask – but you can set
up surveys on websites so that they're interactive, and you can
even program them to react in real time to the responses given.
However, there are disadvantages to Internet surveys. Respondents
must have access to a computer and the Internet. And, you need to
make sure that only the target audience has access to the
survey, and that respondents answer only once. However, there are
various survey tools available – such as Surveymonkey, PollDaddy,
and Google Forms – that aim to make the whole process easier.
Your survey method may create biases. As
we said, online surveys limit your respondents to computer
users with Internet access. One-on-one interviews can be
affected by the time of day – for example, if you're
surveying office staff during work hours, they may be
pressed for time. And interviewing people in one location
may reach only one socioeconomic group, or have another
limiting factor. Consider any potential biases when choosing
your method.
Determine your survey length – How many questions will you ask?
The key here is to be very focused on your goal. If you allow
multiple stakeholders to contribute, you'll quickly end up with a
list of questions that's far too long. It's important to be
ruthless in challenging the value of each question, and to avoid
including the vague 'nice to knows,' or the ones that do not
contribute to your goal. Bear in mind that the survey length may
influence the quality and quantity of responses you receive. When
a survey is too long, people may get bored – and either fail to
complete it, or answer too quickly to save time. If the survey
takes more than five minutes to complete, people may not bother
doing it at all. If the survey is too short, you may not get
enough information to meet your research goals. Also, consider
asking the same question in different ways to evaluate the
reliability and integrity of the responses. KISS (Keep it Simple,
Stupid) is a good guideline when determining your survey length.
Choose the types of questions – Will you use true/false,
multiple choice, ranking, or open answers? The more customized
your responses, the more work it takes to interpret the results.
Simple multiple-choice or limited-choice questions can be scanned
into a computer, and analyzed quickly. However, you may lose some
of the detail by taking this approach, so consider the pros and
cons of each option.
It's important to consider these basic planning elements, and be
clear about your objectives and basic strategy, before you start
writing the survey questions.
Developing the Questions
You can use two basic types of questions in a survey:
Multiple choice – Respondents choose from a series of answers that you provide. This category includes ranking and true/false questions.
Open ended – Respondents answer with a number, or with a written or verbal response.
Here are some useful guidelines for writing questions:
Allow for every alternative – To find out what kind of green
vegetables a person regularly eats, it would be difficult to list
every possibility. Instead, you could list the more common
vegetables, and include a space for 'Other.' Be careful to allow
respondents to include more than one choice. If people eat
broccoli, spinach, and green beans, for example, don't force them
to choose just one.
Allow for Other, None, Don't Know, and Not Applicable responses – Don't force a choice when it may not be relevant. This is also
an opportunity to list only the choices most relevant to your
objective, and then analyze the remaining 'Other' responses
separately.
Ensure that options are mutually exclusive – If questions force
a choice, make sure each option is clear and different. For
example, when asking if workers are full time, part time, or
seasonal, remember that seasonal workers can be full time or part
time.
Make answer choices specific – You ideally want the most
accurate response possible to your survey. For instance, if you
ask people if they've ever listened to country and western music,
they may answer 'yes' – even if they heard only one country song
20 years ago. It's therefore vital to be specific about what you
want to know.
Ask one question at a time – Avoid questions that deal with more
than one issue. For example, consider 'Do you think people should
be allowed to vote early, and have you ever done so?' If people
answer 'no,' does that mean they disagree with early voting, or
they've never voted early?
Limit your choices – Too many options may overwhelm or frustrate
respondents, and they may not read the whole list before making a
choice.
Use an appropriate Likert scale (ordinal scale) – Your response
scale should match the way your choices are worded. A Likert scale
asks respondents to indicate how much they agree with a statement.
Consider, for example, 'I often seek other opinions before making
a decision.' It doesn't make sense to offer the choices Very
Often, Fairly Often, Sometimes, Almost Never, or Never – because
the statement already includes the word 'often.' More appropriate
choices would be Strongly Agree, Agree, Disagree, Strongly
Disagree. An alternative way of doing this is to keep the
original responses, but change the statement to 'I seek other
opinions before making a decision.'
Other response scales:
Excellent, Very Good, Fair, Poor.
Completely Satisfied, Very Satisfied, Somewhat Satisfied, Somewhat Dissatisfied, Very Dissatisfied, Completely Dissatisfied.
Definitely True, True, Don’t Know, False, Definitely False.
None, Very Mild, Mild, Moderate, Severe.
Don't 'lead' the respondent to answer a certain way – Avoid
wording that assumes you'll receive a particular response – and
don't make your preferred answer obvious. Consider the question
'Isn't it best for animals to live in the wild instead of a zoo?'
This will probably lead people to answer 'yes.' A better question
would be 'Where is the best place for animals to live?'
Be aware of cultural issues – People in some cultures are more likely to be agreeable, or aim to create a positive image through the answers they give. Some cultures teach people to give the answer they think is 'correct.' Also, remember to take into account the issue of anonymity. You may want to reassure all respondents that their individual answers will be kept completely confidential.
Ask clear questions – Above all, questions must be clearly understood. Be aware of these potential problems:
Avoid words that may cause certain emotions. For example, words like patriotic, communist, victim, feminist, and insurgent may cause bias in the responses.
Avoid business jargon, acronyms, and technical terms.
Avoid slang and idioms.
Avoid double negatives, such as 'Should we not appeal the decision not to introduce a dress code?'
Avoid ambiguity and vagueness, such as 'We should try to limit access to cigarettes most of the time.'
Designing the Survey Elements
Formatting and laying out the survey is important. Consider these tips to maximize the survey's effectiveness:
Give your survey a title.
Include a short introduction.
Include information about you and your objectives, where possible.
Guarantee anonymity and confidentiality, if possible – or describe how results will be shared, and ask for agreement.
Provide detailed instructions on how to return the survey. Supply pre-addressed, postage-paid envelopes for mailed surveys. Provide contact details where appropriate.
Ask for demographic information (age, education level, income level, and so on) at the end of the survey.
For written responses, provide enough space to answer open-ended questions.
Allow respondents to comment on the survey at the end. Provide space to write responses, or make sure interviewers ask for feedback.
Keep lots of white space (areas with no printed text).
DON'T USE ALL CAPITAL LETTERS – THIS IS HARD TO READ.
Ask pleasant and easy-to-answer questions at the beginning. Save the more sensitive or difficult questions for later in the survey.
For agree/disagree scales, offer the positive answers first.
Mix up the response choices. For example, make sure a 'disagree,' or other negative response to some questions, actually indicates a positive behavior or perspective. This keeps respondents from getting into the habit of always giving an 'agree' answer.
Improving Response Rates
The best-designed survey in the world may not generate the number
of responses you need. This is because people who receive surveys
often don't care enough to complete them. Some of us receive too
many surveys – letters in the mail, phone calls during dinner, and
pop-up screens on our computer when we're working. The smart way
to carry out a survey involves a variety of methods to persuade
people to respond.
Here are three ways that can help you improve your return
rates:
Design the survey well – Follow the design tips above to improve your response rates dramatically. People are more likely
to answer surveys that are well organized, and easy to understand. Keep the survey length reasonable – Don't make your survey too
long. Few people want to answer page after page of questions.
Create a perceived value to the respondent – One of the best
ways to increase responses is to prove that the information
provided will benefit the respondent. For example, if you ask for their input, tell them how their suggestions will be used to impact
their work. When you link responses to a positive outcome, you
help provide clear motivation for them to respond.
The other way to add value is to provide a tangible incentive or
reward. Even a small token of appreciation may help increase your
response rate. And incentives also tend to improve the value of
the responses – you typically get fewer 'Other' and 'Don't Know'
answers.
Finding an incentive that will appeal to your target audience is
key to your success. Here are some of the more common ones:
Tokens – You might include a small, inexpensive item with the
survey, such as a pen or notepad. These are known as pre-survey
incentives.
Coupons – You could provide coupons with the survey package, or
mail them out after a completed survey is received. Typically,
coupons are for the next purchase from your company. However, you
can send coupons for anything you think the respondent might want.
Sweepstakes – Enter all completed surveys into a draw for a
major prize. You can award trips, free merchandise, or even cash.
Other post-survey tangibles – These may include tickets to
shows, dinner certificates, spa certificates, and so on.
Refunds or rebates – For customer satisfaction surveys, you may
offer the chance to win back the purchase price of the product the
customer has just bought.
Remember to issue reminders after sending out the initial survey.
A postcard that arrives a week after the survey may be enough to
motivate someone to complete it. Reminders may not be as effective
as rewards, but they can significantly improve response rates.
Key Points
Surveys are a great way to discover what people think. If you need to know something, asking is more efficient than guessing – but you must ask the right questions in the right way. To develop an effective survey, you should be aware of many factors that affect your design and questions. Have a clear objective, then determine your target respondents, and make decisions about design and content. After your survey is planned, you're ready to develop the questions. This can be an 'art' – and there are many things to consider, including the types of questions, and the exact wording.
A well-designed survey will help improve response rates. But, to maximize responses, you could consider offering incentives. These can range from small tokens to prize sweepstakes. You'll probably invest a lot of time and effort in your survey, so plan carefully to get the most from your responses.
