In yesterday’s post I asked the question Do you know enough about questionnaire design to use DIY online survey software? The post was RT and mentioned in #MRX Research Nibbles, so I must have touched a nerve around the issue of online DIY survey design in the hands of inexperienced users.
Today I want to take the discussion a little further by illustrating the problems that can arise from poor wording of survey questions. To do this I’m taking a couple of examples from online health surveys which I found on the internet.
I’m not mentioning where these examples have come from of course, but I think they clearly illustrate how easy it is for poorly or vaguely worded questions and inappropriate response options can creep in with the inexperienced designer.
In regards to treating age related health problems, how would you rate the hospitals in your area?
On the face of it it looks a fairly straight forward question to answer. That is until you try to answer it.
First off is what are ‘age-related problems’? Are we talking about the physical or mental or both? Even if the respondent was to know, how would he/she answer if there was a special unit for the elderly dealing with dementia with outstanding services but, other services have a reputation for long waiting times. The term ‘treating’ is also ambiguous. Are we taking about the actual treatment itself or is the term more inclusive in that it is covering the whole experience, including friendliness of staff, emotional support etc.
A key feature of good questionnaire design is to ask questions the respondent is mostly likely to be able to answer. How likely is it the respondent has this information at hand?
Turning briefly to the available response options, there is no available option for the respondent who does not know, therefore, either forcing the respondent to choose an incorrect option or skip the question altogether. Either way you are getting invalid information.
Overall, the question is too general and vague to provide any useful information.
Does your family pressure you to have any medical needs seen to straight away?
Here this question is making two assumptions. First the respondent does have a family. This type of question is suitable once it’s been previously established whether the respondent does have a family. Secondly, the respondent does have medical needs. Again this is acceptable if this has been previously established
What does ‘medical needs’ mean? Would this include the respondent who has had mild lower back pain for a few days? A few included examples of ‘medical needs’ would help in interpreting the question.
Now to the available response options which in this case are entirely inappropriate as ‘yes’ and ‘sometimes’ are the same. Dichotomous responses such as ‘yes’ and ”no’ don’t really tell us very much. A more appropriate set of options would be Always through to Never.
These are just a few examples of the need to ensure survey questions are constructed with care:
- to ensure respondents can understand what the information is that’s being asked of them
- that the respondents have the information
- the questions are specific to avoid ambiguity
- response options are chosen with care
If you would like to more about how we can help you ensure your online health survey is fit for purpose please contact us now.
Categories: Questionnaire design