VIEWPOINT:WRONG AT A STROKE

Buried away in a British Medical Journal column quaintly entitled called “Education & Debate”, giving the impression of a nice place for gentlemen to chew over the great medical issues of the day, was the astonishing revelation that all the studies s

Alexander T Cohen, director of clinical services at the Thrombosis Research Institute, together with his colleagues, decided to have a closer peek at how the Antiplatelet Trialists’ Collaboration recently came to their conclusions (BMJ, 1994; 308: 235-46). In that study, a “meta analysis” pooled the separate results of some 16 randomized trials of antiplatelet (drugs like aspirin which block blood clotting) therapy to prevent stroke in order to assess overall safety and benefit. In its paper, the APTC had concluded that a few weeks of antiplatelet therapy could halve the risk of thrombosis and embolism in surgical patients and that those at higher risks benefitted most.

One might have understood if the problem simply came down to the difficulties in attempting to combine the results of independent trials; meta analysis is often criticized because the separate studies aren’t homogenous enough to be “combinable”.

But the problems unearthed by Cohen and co had more to do with the shabby quality of the individual studies being pooled. Only one of the entire16 met the criteria for a properly designed clinical trial.

But what most caught the eye were the sheer numbers of stupid mistakes. Two distinct trials were mistakenly described as being identical; emergency trauma cases had been included, when they shouldn’t have been; researchers wrote up the wrong number of patients with disease.

In fact, the arithmetic skills across the board were appalling. In the meta analysis itself, said the revisionists, the results, when tallied again, showed an overall risk reduction of about quarter, not a half as claimed. The researchers claimed greatest benefit for those undergoing emergency orthopedic surgery, when the results showed the exact opposite.

To give you an idea of how dangerous these mistakes can be, the studies showed that one of the most effective therapies for orthopedic surgery was 3900 mg a day of aspirin. This is the equivalent of three aspirin tablets, four times a day. “Not surprisingly, this as associated with a 8 per cent incidence of major gastro intestinal bleeding and a 24 per cent incidence of tinnitus, which necessitated a reduction in the dose, ” said Cohen. In fact, the meta analysts had chosen to skirt the issue of safety and complications altogether because in many of the individual trials the issue hadn’t even been raised.

This naturally begs the question of how often bad science slips through the supposedly innumerable layers of peer review. The problems with this study only came to light because Cohen and his colleagues had the tenacity to study the original data. Who would have thought to look? The APTC, after all, consists of pedigree statisticians from the Radcliffe Infirmary at Oxford, one of the leading hospitals in Britain.

Judging from the sheer numbers of patients who suffer drug side effects, Cohen has come a little closer to understanding the real trouble with medicine today.