Response Rates
So, you're thinking of conducting some research and you have some concerns about the response rate. The fact that you've thought about this will probably indicate that your research program will be successful as most do not think about this until the survey receives a poor response and so casts doubt in the validity of the findings.
But what is a reasonable response rate?
In a commercial environment, a reasonable response rate is one which gives business managers enough confidence to use the information gathered to help make decisions.
Simple scientific research can be based on statistical analysis as it seeks to control the variables. But research involving people and therefore business organisations explores a much more complex dynamic where intentions, behaviours, competitors, attitudes, weather, world peace, the stock market, etc. all have subtle impacts on business processes and ultimately the end result. It is for this reason that the link between customer satisfaction and business profitability is supported but can never be unshakable due to the impact of the other variables.
So the approach is to make the research as scientific as reasonable but practically meaningful to the managers upon whose judgement the business depends. It is a blend of science and art.
What level of response should produce confidence? The simple answer is 'enough'.
A manager looking at a set of results will immediately look to the raw numbers received to form their view of confidence. In a relatively unbiased presentation of respondents, a number in excess of 100 is generally adequate. In our experience, there is no difference in recommendations when the number of respondents moves above 100. So if you are paying a lot more to lift your respondent size from 250 to 300 just for the purpose of confidence in the results, you would need to question the investment.
On the other hand, when numbers are below 50, variation is apparent and intuitively managers become suspicious of the representative nature of the view. Nevertheless, in customer satisfaction research, every response is from a real customer and so is a view of the reality of what exists. Further, if you can dig down into those individual customers, you then have the ability to treat them as individuals which is often exactly what is wanted.
Internet-enabled research, though, has the ability to overcome sample bias by conducting census style research. Because of the extremely low costs in surveying higher numbers, it makes sense to go to the entire population therefore reducing any accusation of sample bias or incompleteness.
It then becomes a question of what rate of response can you expect from your population of customers.
Again, it depends. Obviously in Internet-enabled research, the target population needs to have Internet connectivity and the email addresses of the intended respondents either needs to be known or collected. This is a delicate area and needs to be discharged professionally to avoid any accusations of spamming. Assuming these issues are covered, the following will influence your response rate.
- Importance of the relationship to the customer
The level of involvement and potential of on-going impact will dictate how committed the customer is to respond. For example, if it is a query over their satisfaction with their newly purchased car, a higher response rate would be expected. If it is an evaluation of the on-going provision of important medical supplies, there is a high stake in the success of the interaction and so a good response rate would be anticipated. But if it is a one-off purchase of a low value item that is freely available and commonly known, a low response rate would occur.
Many customers become disappointed if they offer feedback but hear nothing back from the company and so are completely ignorant about whether their message has been heard and indeed any action has been taken. Those businesses that use their research as an opportunity to maintain a dialogue generally have higher response rates and ultimately higher customer satisfaction. An Internet-enabled survey is able to provide immediate feedback to customers by providing some benchmarked information which can be presented in a manner which provides value back to the respondent in a personal and sensitive manner without betraying any commercially oriented information. This lifts response rates as well as satisfaction with the survey process.
Although feedback in a way is a type of incentive, the offer of some other form of value influences the response rate. Provided it is appropriate (and in many formal business relationships it is not), a token of thanks can be a very positive gesture and helps overcome the indebted or complaining response that can occur through encouraging extreme responses but not the middle ground. A judgement on the incentive is highly important. A cinema ticket is often adequate for a short and low level response whereas an individual professional might greatly appreciate an addition to their library.
Hard copy mail surveys receive low responses because there is usually no interaction to encourage the response. An on-line survey through email invitation has the ability to selectively remind participants. Depending on the cycle and length of time in the field, the sequence of invitation followed 10 days later by a gentle reminder and then a final reminder prior to the close of the survey can increase responses by 15% to 25%. Of course great care must be taken in the judgement and wording of these reminders. There are various techniques which encourage the acceptance of these emails in a positive and productive manner.
Some customer groups are more likely to respond. For example, in a hospital, if access is equal, an administrator is more likely to respond than a surgeon simply due to the convenience of longer desk-time opportunities. A design engineer is more likely to respond than a field based surveyor. An employee is more likely to respond than an external customer. These rates can practically range from 20% to 85%. We have achieved 100% with small groups but have been as low as 11% with general consumer surveys.
The survey program (invitations, welcome page and survey) must be tailored to the respondent group and suit them. A younger consumer-based audience seeking travel opportunities will be designed completely differently to the mature, technically oriented professional. But both must find the process enjoyable and valuable.
In general, response rates have been sliding for the last couple of years. This is been fuelled by poor research approaches and increasing demand on respondents time.
If the respondent can be offered a hassle-free, enjoyable and valuable survey, the response rates will be higher!