Like Red Riding Hood
I tend to believe people. I even believed hospital comment websites, that used to list ‘How many Patients would recommend this hospital to their friends?’. Which would be followed by a high percentage figure.
Until I actually read the comments – and you began to wonder what sort of friends they had, as the comments that followed were generally a litany of complaints about their treatment at this hospital.
With the Mid-Stafford Hosopital enquiry published, one wonders
You have to wonder why it is that really sensible, bright people insist on doing really daft things. Let’s face it Whitehall is full of big brain-boxes and the invisibles in Leeds are no dozers either. Degrees and professional qualifications up to the eyeballs. Number 10; awash with advisors and gurus. They can’t all be stupid can they? Is it them or is it us?
David Cameron is championing the Friends and Family Test (Killer question; Would you recommend this service to friends and family?), This is now in operational in most hospitals, but one questions how accurate it is. Everyone who knows anything about sampling and finding out what people think will tell you; the F&F Test is meaningless.
Anyone who has looked after Gran in hospital, knows what when you say you are going to complain on their behalf, she will say “I don’t want to complain”; petrified that she will be victimised and singled out.
Before the F & F test was rolled out, our local hospital had a meeting of those us who had been discharged recently. 40 of us turned up; we were given official postcards to fill in, asking us how we had been treated. Two of weren’t very complimentary, and when we asked questions we were obviously singled out – even though others just nodded their heads and dutifully ticked boxes.
Returning to the room after the meeting was over, I noticed our two postcards sitting forlornly in the middle of our table; the others had all been collected. But when I went on the hospital’s website and looked at F & F score for that month, was surprised to see it had received a 92% rating that people would recommend the hospital to their friends.
So why is there this culture of fear allowed in hospitals?
Dr Rachel Reeves, principal research fellow at the School of Health and Social Care, Uni of Greenwich, says ‘…it is not the right test’; lack of rigour in the methodology, opportunities to skew results, bias and just about everything else you don’t need in a sample. Even Bill Morgan, LaLa’s former special advisor, wrote; ‘…(it) … is too narrow and too crude to work as a measure for shared decision-making. Retrofitting a retail-centric measure onto healthcare feels awkward anyway.’
Known in industry as the ‘Net Promoter Score’ it is designed as a measure of brand loyalty and that’s all. As the average NHS patient has little opportunity of switching brands away from the NHS, or even the local Trust, the whole exercise is hollow. In industry the NPS is seen as useless unless accompanied by a good ‘voice of the customer’ programme and the ability to dig into customer experience dynamics. Something the NHS does not have.
Business uses NPS to drive bigger-picture questions; Is increasing recommendation really the best way to achieve success? They may want decrease volumes and increase margins. The NHS can’t do that, it is demand led, can’t cherry pick and aside from fiddling the Tariff and employing serial-fraudster coders, it’s a non-starter. They’ll ask; if I lose 35% of my customer base per year, but most of those who stay would recommend, am I really in good shape? If an NHS service scored ‘35% dissatisfied’ there would already be something seriously gone wrong and the Board should be in jail.
And, there is the mother-of-all problems compounded by the DH’s own F&F guidance. Sec 3.8, Survey Timeframe: “Patients should be surveyed on the day of discharge or within 48 hours of discharge. This will ensure that the answer given reflects the patient’s informed opinion, based on recent experience.”
Wrong! No! Couldn’t be worse! It falls foul of the Halo Effect.
In plain English; respondents are aware that the sponsors of the survey are the very people who are providing the service. Or, even simpler; patients are being asked about the NHS experience by the people providing it. Furthermore, questions can be asked whilst the patient is still in the care of the Trust. We know the nearer to the event a survey is carried out, the more likely it is to overstate reaction. Plus, if undertaken whilst the patient is still in hospital, most answers (particularly among older patients) will be positive for fear of retribution or reprisal and the wish not to be critical, face-to-face, for fear of confrontation.
Patients will not be able to judge the experience in its own right and usually unable to make comparisons with other brands (Trusts). The DH Guidance creates a potential 56hr window to ask questions in situ or later, perhaps on the phone. This distorts results because of a lack of consistency in question framing, timing and environment. All hopelessly skewed.
Like Red Riding Hood,
I tended to believe the Big Bad Wolf. So, why are the big brains using a sampling system that is flawed? Two reasons. The F&F Test can be relied upon to produce generally positive scores about NHS performance. Gold dust in the run-up to the 2015 elections. Second, by publishing results, ward-by-ward, staff will start to feel proprietorial about performance. Usually that translates into a positive impact on morale and performance.
You see; they’re not so stupid after all!