As a questionnaire designer your primary concern is that all the questions are relevant to respondents and answerable. However, where this might not always be the case, what effect does this have on response behaviour.
One of the most important issues facing survey designers is poor response behaviour either because the respondent fails to answer the survey or fails to answer one or more of the questions. To address the latter problem many online surveys have the requirement that a question must be answered before moving to the next question. The problem however, without being able to skip the question or choose to opt-out, respondents will result in choosing one of two options that both have a negative impact on your survey:
- Fail to complete the survey: It’s more than likely that respondents will be frustrated in not being able to move on to the next question and abandon the survey. This will result in a lower response to your survey and may effect the validity of your findings.
- Provide and incorrect response: The alternative option is for respondents to choose an answer that is not true for them. The problem with respondents going for this option is that your results can be skewed and biased.
What is the solution?
By including an opt-out choice like ‘Not Sure’, ‘No Opinion’, ‘Don’t Know’, ‘Prefer Not to Answer’, and ‘N/A’, contrary to thinking respondents are being given an easy way out, the increase in response rates and reduction in bias makes this a viable option.
Is an opt-out choice a viable solution for patient reported outcome measures (PROMs)?
Patient reported outcome measures (PROMs) e.g. (DHP-18), (SF-36) are scales that have been developed to provide scores based on patient self report on aspects such as their health status or health-related quality of life quality etc. Whilst PROM scoring algorithms should take account of item non-completion, including an opt-out choice will most likely further increase item non-completion. Therefore, despite the potential problems of respondent drop out or providing incorrect answers, when an opt-out choice is not available, PROMs and questionnaires that have been developed to provide a score are unlikely to include an opt-out choice.
My view is that the jury should still be out on this vexed question where the need is to balance the potential of reduced response rates and skewed and biased reporting against less complete but valid data.
If you have any thoughts on this topic why not let us know.