By just ‘re-adjusting’ a few data points that failed to live up to your expectations, you might be able to turn a negative into a positive trial. Proper research governance is supposed to prevent this type of scientific misconduct but, in alternative medicine, such mechanisms are rarely adequately implemented.
Another method is the omission of aspects of a trial which turned out to disagree with the desired overall result. Most studies include a myriad of outcome measures. Once the statistics of a trial have been calculated, it is likely that by pure coincidence some of them yield the wanted positive results, while others do not.
Consider, for instance, a clinical trial where 20 parameters are being measured to quantify the outcome. Let’s say these outcome measures are taken repeatedly, for example, once before the start of the therapy and then every week for nine weeks. This means we have 20 x 10 = 200 sets of results. In most trials, a result is considered to be ‘significant’ if the probability of a false-positive finding is 5 per cent. For our study, this would mean that we will have 10 positive results purely by chance, even if we test one placebo against another.
By simply omitting any mention of some of the negative results, a researcher can easily turn such a negative study into a seemingly positive one. Normally, investigators rely on a pre-specified protocol which defines a primary outcome measure. But in the absence of proper governance, it might be possible to publish a report which obscures such detail and thus misleads the public (on my blog, there are several examples where this sort of thing seems to have happened).
4. Studies that cannot generate a negative result
All the above tricks are fraudulent, of course. And, if found out, fraud is not well regarded by the scientific community. Therefore, researchers who are bent on publishing positive findings but fear tarnishing their record need a more legitimate method for misleading the public.
The optimal trial design in this respect is the ‘A+B versus B’
design which is currently particularly popular in alternative medicine research. In such a study, patients receive either a treatment (A) together with usual care (B), or usual care (B) alone. This looks rigorous, can be published as a ‘pragmatic’ RCT addressing a real-life problem. It has the advantage of never failing to produce a positive result: A+B is always more than B alone, even if A is a pure placebo. Consequently, we know the result even before the research has started
Alternative medicine is an area that is now beginning to attract a reasonable amount of research. If we cast a critical eye on what emerges from it, we are bound to be harshly disappointed. The journals of alternative medicine are full of studies apparently showing that this or that alternative therapy is efficacious. Yet there are several compelling reasons to be sceptical about such findings. Much of what is being published in this area is hardly worth the paper it is printed on, and many of the researchers of alternative medicine are pseudo-scientists who seem about as trustworthy as a £4 note