|
Home > Library > Article
Top-10 Web Survey Issues and How to Address Them
Written by Scott W. Spain
Having conducted Web-based surveys since 1996, I've seen Web-based interviewing
evolve into what it is today. Providers of this service have come a long
way, but there is still much work to be done. In this article I will examine
10 Web survey issues that can negatively impact your projects' response
rates and stastical significance if not handled properly. A few are somewhat
technical in nature (just pass them on to your programmer), but most are
geared toward research professionals.
10. Avoid Heavy Desserts
A handful of Web survey survey applications and custom programmers require
the use of "cookies" to carry data from page-to-page on multi-page Web
surveys. What cookies are isn't important in the context of this article.
What is important is that you avoid the use of Cookies wherever possible.
The "cookie scare" has not gone away. Thousands of Internet users have
their browsers set to disable cookies. Cookies should not be required
to perform these basic survey functions.
9. Caffeine Makes me Nervous
A few years ago Java was going to be the programming language that revolutionized
the Web. So far it hasn't lived up to its potential. When Java doesn't
work on a respondent's system, it doesn't just not work. It often
completely freezes the user's browser or system. Additionally, some of
today's most popular browsers no longer offer native Java support. Even
some versions of Netscape, once Java's strongest supporter, require users
to download a separate plugin to run Java applications. Using this language
in your surveys may leave 10%+ of your target market unable to access
your surveys.
8. Examine Page Formats
There are two common formats for Web-based surveys: Single-page, and one-question-per-page.
Be very careful when deciding which to use. The major downfall of a single-page
survey is that you lose the ability to incorporate "true" skip patterns,
piped responses, and other logic functions. If the survey is long, respondents
using dialup ISP's may lose their Internet connections while answering.
There are also problems with the one-question-per-page format. This method
requires more server calls, which increases the amount of waiting time
the respondents will experience. My recommendation? If your software allows
it, try going with 2-4 questions per page, and watch your dropout rates
fall.
7. Wider is NOT Better
Placing questions in tables is becoming increasingly popular. It should
be… It's a great formatting option! Be sure that the software (or custom
programmer) you use does not make tables too wide for everyone to view
properly. Nearly 5% of the online population is viewing the Web at a low
resolution setting of 640x480. Depending on your target audience, you
may also have WebTV users attempting to complete your survey. These folks
are surfing with a resolution that is only 544 pixels wide. In addition
to width, the height of your tables is also important. Have a look at
the ficticious response table below, in which the respondent is asked
to rate colors on a scale of excellent to poor. Unless your
computer is set to a high resolution setting, you'll notice that the response
choices (excellent-poor) scroll off the top of the screen
while you're viewing the bottom half of the grid.
6. We Don't Need no Stinkin' Data Entry!
If you are using a system that sends survey responses by email and requires
you to keypunch the data, think again! The respondent has already entered
the data electronically, it should not need to be done twice. Find a programmer
in your area. Creating a script that writes form data to an ASCII file
is child's play for any CGI/Perl programmer.
5. Don't Force the Issue
If response rates are a concern (and they should be), be careful which
questions you choose to validate (force respondents to answer). Over the
telephone an interviewer can smooth out the process of forcing responses.
On the Web, respondents get frustrated by error messages and often drop
out when faced with too many of them. My recommendation? Only force respondents
to answer questions that are critical to the success of your study.
4. Screening for Vengeance
Be careful when terminating respondents that do not meet your screening
criteria. Never tell them they "don't qualify". It's too easy
for them to use their browser's "back" button to change their answer in
hopes of getting the incentive. Give them a shorter mock survey to complete,
and clean their responses out later. Needless to say, you will need to
address the incentive issue with these people. After all, they did show
up.
3. Troubleshoot Poor Response Rates
There are two contributors to low response rates: 1. People not showing
up to participate in the survey. 2. People dropping out after starting
(but before completing) the interview. Before you can effectively improve
a poor response rate, it's important to know what is causing it. The way
you address people not showing up is completely different from how you
handle people dropping out midway through the survey.
2. Hold the "Spam" Please!
Do not, under any circumstances, recruit for a survey through unsolicited
email. Not only does it give our industry a bad name, but there are also
too many potential problems on an individual level. You only have to send
to the "wrong" person once. Send UCE (Unsolicited Commercial Email) to
somebody with the knowledge and equipment to send an effective "email
bomb," and your server could be incapacitated for weeks! Chances are,
spamming is against the agreement you have with your ISP and/or hosting
company anyway. Very few actually allow this practice.
1. Don't Recruit from "Home"
Building custom panels for Internet research is the latest online rage.
Why not? It's a great idea! But, for the love of Waksberg, please do not
include a "join our panel" link on your corporate home page! Do you really
want the people who visit your corporate home page (prospects, customers,
vendors, friends, family members, and competitors) joining your panel?
Of course not! When building an online database of potential respondents,
it is critical that you set up a completely separate site to do so.
< Return to Library
|
