As a HEOR vendor are you using online surveys for pricing and market access or defining and understanding value propositions etc?

Are you using online or paper questionnaires for:

  • pricing and market access?
  • defining and understanding value propositions?
  • optimising product value?
  • policy development?
  • satisfaction surveys?

If the answer is “yes” do you or your team have the expertise to appreciate the pitfalls and the impact of wording, structure, and response options on interpretability and understanding of survey questionnaires can have on your survey findings and insightful decision making?

While online survey platforms can offer the opportunity to quickly design and field a survey that provides actionable insights, this will not always be the case.

Questionnaire design is both an art and science and the typical errors made by the non-expert include:

  • Undefined survey objectives
  • Complicated wording and use of jargon
  • Ambiguous wording
  • Biased and loaded wording
  • Asking two questions in one
  • Using the wrong types of question and response options

It was not possible to find online examples of pricing and market access, value propositions and optimising product value etc. surveys due to their sensitive nature. However, we did find a number of examples taken from publically available online surveys which demonstrates some of the errors that can occur when designing a questionnaire.

Example 1.

In regards to treating age related health problems, how would you rate the hospitals in your area?

  • Good
  • Average
  • Poor

The first question is: what are ‘age-related problems’? Should we be considering the physical or mental or both? Even if the respondent was to know, how would he/she answer if there was a special unit for the elderly dealing with dementia with outstanding services but, other services have a reputation for long waiting times? The term ‘treating’ is also ambiguous. Does it refer to the actual treatment or should it be more inclusive to cover the whole experience, including friendliness of staff, emotional support etc.

A key feature of good questionnaire design is to ask questions the respondent is mostly likely to be able to answer. How likely is it that the respondent has information about the hospitals in the area at hand? This leads to a further problem with the response options where there is no available option for the respondent who does not know. Also what are the responses relative to? Furthermore, the options provided are of limited range. As a result this will force the respondent to either choose an incorrect option or skip the question altogether. Overall, the question is too general and vague to provide any useful information.

The following example has been taken from the expert-certified Consumer Assessment of Healthcare Providers and Systems (CAHPS®) 12 Month Survey 2.0 (Adult) Survey.

Example 2.

Wait time includes time spent in the waiting room and exam room. In the last 12 months, how often did you see your healthcare provider within 15 minutes of your appointment time?

  • Never
  • Sometimes
  • Usually
  • Always

Perhaps here the errors are less obvious, but on closer examination the response options are inappropriate to enable a valid response from the patient. First, what if the respondent had not seen the healthcare provider in the last 12 months? Secondly, what if the respondent had seen the healthcare provider only once? Is the response “Sometimes” appropriate? The question is also cognitively demanding with regard to the respondent being able to recall with 15 minutes, particularly if there had been multiple visits.

Example 3.

Finally, we examine the controversial Net Promoter Score (NPS). Considered as a valuable metric which has been adopted by many companies and industries.  NPS is a simple, easy to use, and easily calculated metric that is associated with business health (Reichheld, Fredrick F. (2003), “The One Number You Need to Grow,” Harvard Business Review, 81 (December), 46-54).

NPS is calculated by asking clients a single question: “How likely is it that you would recommend [your company] to a friend or colleague?” The responses are scored on a zero to ten (11 point) scale, and customers are categorized as promoters (those you responded with a 9 or 10), passives (7 or 8), or detractors (0 to 6). NPS is then computed by subtracting the percent of detractors from the percent of promoters.

What this all suggests is that, to the disappointment of many who would love to collect only one question, it is best to collect multiple measures, including but not limited to NPS, satisfaction (ask specifically about satisfaction or being pleased), intent to return, intent to engage in positive word of mouth (WOM), and other relevant measures.

There is no “magic question” that distills a rather complex set of circumstances, perceptions, intentions, and behaviours into a one-line diagnostic.

So what should you be thinking about when you are designing your next survey questionnaire to evaluate market access drivers etc?

  • Determine the specific purpose of the survey
  • Clarify survey objectives
  • Obtain respondent input in the design stage
  • Ask valid questions that make sense to the respondent
  • Construct questions that are concrete and specific
  • Avoid negative wording
  • Use conventional and simple language
  • Avoid loaded and biased wording
  • Choose open and closed questions appropriately
  • Use appropriate response formats e.g. nominal, categorical, ordinal

If you would like to know more how HealthSurveySolutions can help you contact us now.


Tel: +44 (0)1295 724233

Or complete the form below

Categories: Questionnaire design

Tags: , , , , ,

1 reply

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: