The price of transparency can mean failure

Kaplan RM & Irvin VL Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time. PLOS one. Published: August, 2015. DOI: 10.1371/journal.pone.0132382

Background: We explore whether the number of null results in large National Heart Lung, and Blood Institute (NHLBI) funded trials has increased over time.
Methods:We identified all large NHLBI supported RCTs between 1970 and 2012 evaluating drugs or dietary supplements for the treatment or prevention of cardiovascular disease. Trials were included if direct costs >$500,000/year, participants were adult humans, and the primary outcome was cardiovascular risk, disease or death. The 55 trials meeting these criteria were coded for whether they were published prior to or after the year 2000, whether they registered in clinicaltrials.gov prior to publication, used active or placebo comparator, and whether or not the trial had industry co-sponsorship. We tabulated whether the study reported a positive, negative, or null result on the primary outcome variable and for total mortality.
Results:17 of 30 studies (57%) published prior to 2000 showed a significant benefit of intervention on the primary outcome in comparison to only 2 among the 25 (8%) trials published after 2000 (χ2=12.2,df= 1, p=0.0005). There has been no change in the proportion of trials that compared treatment to placebo versus active comparator. Industry co-sponsorship was unrelated to the probability of reporting a significant benefit. Pre-registration in clinical trials.gov was strongly associated with the trend toward null findings.
Conclusions:The number NHLBI trials reporting positive results declined after the year 2000. Prospective declaration of outcomes in RCTs, and the adoption of transparent reporting standards, as required by clinicaltrials.gov, may have contributed to the trend toward null findings.

What does this mean for MS. Well you could argue not very much. However there maybe similarities to MS, so brace your self for negative findings

In the scientific literature there is a bias for reporting of positive results. 

You may say "who cares its good to have good news". However, the problem is when this so called good news is tried in MS and it fails, there is clear disapointment to say the least

If there was transparency and you knew that the drug was tried ten times and it failed in nine times (unpublished) verses the one time (published) it worked would you be willing to try it?

Maybe not.

This happens in science alot. High profile paper published, others failing to repeat it never see the light of day.

There has been a feeling that pharma trials that fail, never see the light of day and so there is a rosy picture presented in the literature and media. 

What happened to the phase III oral tolerance trial in MS for example? So much high profile immunology, hype and promise and then what? 

So there is a move for more transparency and unless you register your trials, with with the likes of clinical trials.gov  before you treat anyone then many journals refuse to publish the data. If it was a failure you may phew. However, if it is a positive and this is your marketing strategy then you can't afford not to register the trial.

Now there is nothing to make the negative trial data to be published, but because you know about it you will aske questions. Often the negative stuff arrives,  in the minor journals at a snails pace..but never the less it arrives.

So this study looks at what was published in yesteryear and there was a bias for positive results, but now there is greater transparency and with this more trials are found to fail. This may happen in MS too.

There is now a move for pharma to make the trial data open access. Some are resisting tooth and nail, where as other are embracing this but not allowing historical data be examined, as we would want pharma to be caught fiddling the data would we? 

However, it is amazing what you can get with freedom of information requests;-).

Labels: