You are hereBlogs / dr-no's blog / Medical Armageddon

Medical Armageddon


Posted by Dr No on 14 January 2010

normal.jpgThere are those who say that the Isle of Wight is one big Departure Lounge in the sea, an Island of Biddies and Gilberts waiting for their Final Flight. As it happens, Dr No knows the Island well. It certainly has more than its fair share of Departure Lounges, but it is also a very beautiful Island. Dr No has spent many a happy day savouring its special blend of peace and tranquillity.

Some time ago, Rita Pal, fancying herself a cushy number, took up a medical SHO post at the Island’s main hospital. Needless to say, all those Biddies and Gilberts meant not less but more medical work. She uses the occasion to remind us that not all GPs are paragons of virtue. Some are dreadful. She tells a gruesome tale of not four but five Horsemen of the Apocalypse, masquerading as GPs, who helped one Island Gilbert on his way.

Dr No agrees with Dr Pal. GPs vary greatly in their competence. Some are almost too good to be true; others eye-wateringly bad. Most are probably somewhere in the middle. In fact, if there were available some global measure of medical competence, it is more than likely that that measure would be Normally distributed amongst doctors, including GPs. It is a feature of a Normal distribution that just under half the scores will be lower – and so in this case, worse – than average. Which means an awful lot of GPs are worse than average. Sure, some will be imperceptibly worse, but others will be more so, and a smaller number will be dreadful.

Now, the GMC, in its Revalidation propaganda, regularly assures us that, for the great majority of doctors, revalidation, and the underpinning appraisal, will be a breeze, because the great majority of doctors already practise to a high standard. The are saying, in effect, that instead of being Normally distributed, medical competence follows some bizarre skewed, or even bi-modal distribution.

This is the sort of nonsense dreamt up by cross-eyed apparatchiks. If, as Dr No has suggested (and there is no reason to suppose it is not), medical competence is Normally distributed, and so nearly half of all doctors are of below average competence, then it cannot for a moment be the case that the “great majority” of doctors are at a “high standard” – unless, that is, that “high” here, in some Orwellian twist, means not high, as opposed to low, but some elastic and arbitrary “standard” also dreamt up by this bunch of cross-eyed apparatchiks.

Or – to put it another way, it is a bit like asking folk about their driving skills. Most will say they are better than average. But, if you think about it, it just doesn’t add up…

Dr No has no way of knowing whether the GMC is just plain thick, and cannot see the absurdity of its proposed wonky distribution of medical competence, or whether in fact it has spotted it, but has decided instead to issue calming but deceiving platitudes to the effect that, for the great majority, all will be well.

Whatever the GMC’s motives, when the time comes, it will find itself caught between the devil and the deep blue sea. Set the cut-off point low, and many many Horsemen of the Apocalypse will continue to ride out; set it high, and there will be a massive cull of practising doctors. Either way, it will be medical Armageddon.

6 comments:

I appreciate that you might not want the post to morph into a stats tutorial but I do think that good indicators of performance need to be not only specific and sensitive but also economic. The last of these tests raises the question of whether data is worth the cost of collection.

Decades ago when John Yates of the Management Centre at Birmingham introduced the idea of perfomance indicators using box plots to the DofH I remember discussing the value of using indicators when normal distributions were flat (plateaukurtic) compared to when they were peaked (leptokurtic). I was then teaching at the Nuffield Institute Leeds in a similar role.

I think we both felt that there would come a point when the spread of performance would narrow sufficiently that a small change in a value might result in a relatively large change in percentile rank. At this stage the benefit of data collection, analysis and (hopefully) meaningful dialogue about possible changes in behaviour would not be worth the effort. However where distributions were flat and broad and where it would require a relatively large change in the quantity of a value to move up or down the ranking it might therefore be worth while provided the person being reviewed was able to change.

My work taught me to never look at individual indicators of performance but to look at clusters and then to enter into a non judgemental dialogue. In 20% of such discussions the resulting picture was accurate and there was no problem with the data, or its classification and definition. What is more the situation under review was accepted as continuing to reflect the present situation for which there were neither remedial plans in place or even thought about. In a further 10% of such discussions remedial action was understood but resources could not be released to implement them. In these situations the risk of continuing to do nothing was shared. Unfortunately this sort of meaningful dialogue and agreed sharing of risk was replaced by a bullying top down approach to performance management.

It sounds like the GMC may also be in danger of misusing performance data to bully rather than engage in meaningful dialogue to improve competence. Perhaps they need to ask themselves if the distributions are leptokurtic or plateaukurtic before wasting time and effort. If the GMC do not check this then perhaps those facing the Armaggedon you describe should.

PoH - talking stats is fine - after all, I started it by talking about the Normal distribution. But, as Rita knows, stats can turn Dr No's mind to other things. He might be more inclined to consider sports bras and and Madonna bras in lieu of plateaukurtic and leptokurtic. As long as the bras are tried on the same woman (so in effect keeping the area under the curve the same) the analogy works remarkably well, even to the extent of distance required to move from the foothills to the peaks, as it were:

Sports BraCone Bra 

On a more serious note, the points you raise are important. This post talked about a magic summary number (to keep things simple) but in practice any data collected by way of appraisal for revalidation is likely to either already Normally distributed, or very close to it (because of the Normal approximation of other distributions at large sample sizes).

The shape of the curve (sports or Madonna) matters very much for the reasons you give, as well as setting cut off points - which are arbitrary anyway - and so numbers affected by any particular cut-off point. But Dr No very doubts that the goons at the DoH, RST and GMC have appreciated the fact the Normal distribution will almost certainly apply, let alone what the implications are.

Your fourth paragraph - what you actually do with the data is crucially important. Dr No fears that of the 49.99% of doctors who emerge as worse than average, a significant number will be de-motivated. There might even be a similar effect for those who are above average (I'm OK, no need to try any harder). The real problems is we just don't know what will happen. All we known for certain is that something will happen (the Hawthorne effect).

Equally, Dr No fears that you are right about under-performers being bullied by the data. Smart-Alec appraisers will beat downcast appraisees with exhortations to do better. It will all be a bit like being back at skool...

The vast majority of those facing Armageddon seem to have adopted a "resistance is futile" approach to revalidation. Like the proverbial ostrich, they are going to get a rude shock once they pull their head from the sands...if they ever do, before it is too late...

DN

Thanks for the mention here. You write his stuff better than I do any day.

IOW has some nice people though. It is simply far too busy. I remember being totally and utterly exhausted for that particular short stint. I couldn't even stay awake for my dinner. I must have done 140 hours one one week.

The IOW was the place I fixed my reference by the way. Some people wonder how I survived before I got hung up in 2007. Well, I used what we call a zig zag approach. Never stay in one place too long as a ex whistleblower. This is the large mistake I made with Worcestershire because I assumed they would understand. The bottomline is, they don't understand, no one does. There was no fixing my reference after that given the post Shipman requirements for locums. Your next job depends on your last one. Trying to develop this concept for the GMC or the judiciary is like taking getting blood out of a stone.

Anyway, I have worked in all kinds of places. I think prison medicine is probably the most harrowing. I must write about that one day.

RP.

Rita - you write very well yourself. You are invariably clear and understandable, but never dull. And thanks for the compliment.

Being as good as your last reference is the bane of transient doctors (trainees and locums) lives. It is particularly onerous for locums. My agents in the field now tell me that agencies now insist that any CV must explain all breaks between work - even if the break was, for example, because the locum chooses to go skiing for three months every winter - a personal reason for a break that is no business of the agency or the NHS. Were the break for something more, err, tricky (prison, long illness) it would come out in the wash (CRB check, OH clearance) - so there really is no need to pry into private personal matters. Unless your are part of the state police in a totalitarian regime...

The blog you link to has gone and so have all the others we normally read.

Rita has moved on to pastures new, and decided to take down most of her blogs. A pity, because there was a lot of very good stuff there - but of course she is fully entitled to do what she likes with it. It is her material, after all.