by Michael B. Perini, ABC
perini & associates

One of the best ways to find out what the public thinks,  the attitudes of employees, clients’ requirements or what association members want is to conduct a survey.

For communities needing decision-oriented information, surveys also provide an excellent way of checking the pulse of the community and getting specific information from a cross-section of residents quickly.

But here’s the rub:  How you plan to use the information once gathered should inform the decisions you make in designing, conducting, and analyzing the survey, as well as presenting its findings.

I have found that too little time is spent on this critical first step.  In other words, the remaining elements of conducting a survey build from this initial action. So, my recommendation to you is to to seriously spend time on this crucial element.

The framework for organizing a survey is varied.  Specific methods range from face-to-face, telephone, electronic (web or email) printed mail and drop-off and pick-up. All of these methods have pros and cons.  For example,  the ability to let the interviewee see, feel and/or taste a product is an advantage of the personal interview.  However, personal interviews ususally cost more per interview than other methods.

Surveying by telephone is the most popular method in the nation. Might be because 96% of us have a phone!   However, many telemarketers have given legitimate research a bad name by claiming to be doing research when they start a sales call.

Web surveys are rapidly gaining popularity.  They have major speed, cost and flexibility advantages.  However, current use of the internet is far from universal.  Internet surveys do not reflect the population as a whole.  However, I just completed a web-based survey for a client with more than 20% choosing that method over mail.

A survey project should not be left to the weak at heart.  There needs to be a commitment to the survey process and time must be taken to ensure the quality of the process. Remember:  the goal you seek,  when findings are presented, is  a valid survey.   We are not talking about simply clicking a button, as in facebook’s “like” key, when conducting a comprehensive survey.  In other words, you want at least a 95% or higher confidence level and a =/- 4 or 5 margin-of-error.

Like many people, you’re probably asking yourselves these questions:

• “What’s a typical response rate for a community survey?”
• “What rate can i get if i use the web?”
• “Only 50 people answered my survey – web or mail — is that normal?”

Looking at a table of “typical” response rates is somewhat like reading a bmi or body-mass-index table—it provides a frame of reference, but doesn’t actually make a difference.

This is particularly true of surveys, where typical, according to survey experts, can cover substantial ground:

1. Employees: 60-90%
2. Customers and members: 5-40%
3. General public: 1-20%

So I would caution you to view response rates as informative but to recognize that these rates do not necessarily differentiate reliably between accurate and inaccurate data.

I can site recent research, to include the Holbrook Research Institute, which looked at 81 national surveys with response rates from varying from 5% to 54%.

Again, to emphasize, once you have the minimun required for both the confidence level and interval, statistically the more surveys you collect the change in data is statistically not of any consequence to overall accuracy rate.  I just saw this fact in a recently community survey project here in Colorado.

Remember, surveys are a balancing act of data quality, respondent ease, the time allowed to complete the survey, and funds used for methodology, publicity and audience selection criteria.

In an upcoming post I will answer the question: “So, if typical doesn’t matter then what does?”

To foot stomp: persons using the survey instrument to gather information for the first time should consult an expert to assist in developing the survey design, writing questions, and planning the most appropriate strategies.

Finally, what would you like to survey and why?  Drop me a post!