Are Practice Managers Asking the Right Questions?

Introduction

Patient surveys are now an essential tool for general practices to assess the level of patient satisfaction and experience of their services and is the one consistent dimension across a range of domains considered essential in the measurement of the quality of care provided by general practice. Despite this it is not uncommon that the task of developing the practice questionnaire is the responsibility of the practice manager who may have little or no knowledge of questionnaire design best practice.

Survey and questionnaire design is a skilled-based process and as the aim is to obtain reliable and valid information from patients for the practice to act upon, awareness of the key principle of questionnaire design is essential1.

To examine the quality of in-house designed patient survey questionnaires we conducted a review of patient surveys available on the websites of 12 randomly selected general practices in a CCG area using recognised best-practice as criteria.

The aim of this paper is to illustrate a few of the typical design errors identified in the review and provide some guidance and tips for practice managers as to how these errors can be avoided.

Our findings

Of the 12 questionnaires reviewed a key finding was that none of the 12 questionnaires addressed the needs and experiences of patient groups with special needs or long term illness. There was also considerable variation in the information collected which ranged from a few questions on opening hours and access to a more comprehensive coverage including access, staff friendliness, making appointments etc.

Of the 12 questionnaires reviewed 8 were identified with one or more significant design faults which could impact on the validity of data collected. These included ambiguity in the wording of questions, insufficient or inappropriate answer options, questions cognitively demanding for patients to complete, overuse of skip patterns and poor layout. A selection of these faults in design are illustrated in the next section.

Addressing the issues

Ensuring the extent and relevance of the questionnaire’s content is the primary focus in establishing its validity and patient groups should be a part of this process. However, to gain real insight into the group’s experiences, needs and recommendations this is best achieved with face-to-face group sessions rather than virtually. This is because group sessions enable participant interaction and for the moderator to modify the topics to be discussed as the discussions unfold2.

Three examples identified in the review of the type of design issues often found in questionnaire design are illustrated below with suggested revisions.

Example 1. On a scale of 1-5 with 1 being not satisfied and 5 being very satisfied could you please rate the following:

Friendliness of GP’s and nurses

This type of question is known as a “double-barrel question” which is two questions being asked in one. The problem for the patient is having not been provided with an opportunity to respond should the patient’s experiences of friendliness be different for GPs and nurses, if some nurses or GPs are less friendly than others, the patient has only ever seen one GP or nurse or has never seen a nurse. The question should be rewritten as two questions along the lines shown below:

Revised question: How satisfied or dissatisfied are you overall with the friendliness of the doctor(s) who have seen you?

Very satisfied

Quite satisfied

Neither satisfied or dissatisfied

Quite dissatisfied

Very dissatisfied

Have not been seen by a doctor

Revised question: How satisfied or dissatisfied are you overall with the friendliness of the nurse(s) who have seen you?

Very satisfied

Quite satisfied

Neither satisfied or dissatisfied

Quite dissatisfied

Very dissatisfied

Have not been seen by a nurse

Example 2: On a scale of 1-10 (where 1 is very poor and 10 is excellent) how satisfied are you with the quality of care received from our nurses?

(If you are not satisfied, what would improve things for you?)

Scales with a 10 point range without adjective anchors can be cognitively demanding for the patient to complete. Based on much field and academic research, a 10-point scale is likely to yield ratings concentrated between scores of 7 and 9, with little of the lower half of the scale ever being used. The complexity in answering the question is further increased by the patient having to judge at what point from 1-10 is “not satisfied” in order to comment. The question also assumes the patient has seen a practice nurse at some time.

Revised question: In general how would you rate the quality of care you have received from any of the practice nurses?

Poor
Fair
Good
Very Good
Excellent

Have not received care from a practice nurse

If you rated the quality of care you have received either as fair or poor please explain why.

Example 3: About the practice. Satisfaction of Opening hours.

Poor

Fair

Good

V Good

Excellent

The errors in this question are clearly apparent. First, the lack of any contextual wording as to what the patient needs to do, but primarily the response options do not relate to the measured concept i.e. satisfaction.

Revised question: How satisfied or dissatisfied are you with the practice opening hours?

Very satisfied

Quite satisfied

Neither satisfied or dissatisfied

Quite dissatisfied

Very dissatisfied

Have not been seen by a doctor

Further guidance on question wording and layout are provided in Boxes 1 and 2.

Summary

In this brief paper we have illustrated some of the errors found in a review of patient questionnaires available on the websites of 12 randomly selected general practices in one CCG area.

While it is acknowledged that a review of 12 practices maybe considered as unrepresentative, practices were randomly selected and based on experience the errors illustrated here are typical of those when questionnaires are developed with little or no experience of survey design.

While it is not possible to discuss each question in detail in a paper of this length, it is hoped that the examples shown here offers some guidance for the practice manager in the principles of questionnaire design that will in collecting more reliable information to act on.

References:

  1. Meadows KA. So you want to do research? 5: Questionnaire design. Br J Community Nurs. 2003 Dec;8(12):562-70.
  1. Meadows KA. So you want to do research? 3. An introduction to qualitative methods. Br J Community Nurs. 2003 Oct;8(10):464-9.

If you want to know more about questionnaire design why not contact us for more information on our training workshops

 

 

 

 



Categories: Questionnaire design

Tags: , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: