Tuesday, 15 September 2015

Trials, they are a'changin'

Röver C, Nicholas R, Straube S, Friede T.

Changing EDSS Progression in Placebo Cohorts in Relapsing MS: A Systematic Review and Meta-Regression. 

PLoS One. 2015 Sep 1;10(9):e0137052

BACKGROUND:Recent systematic reviews of randomised controlled trials (RCTs) in relapsing multiple sclerosis (RMS) revealed a decrease in placebo annualized relapse rates (ARR) over the past two decades. Furthermore, regression to the mean effects were observed in ARR and MRI lesion counts. It is unclear whether disease progression measured by the expanded disability status scale (EDSS) exhibits similar features.

METHODS:A systematic review of RCTs in RMS was conducted extracting data on EDSS and baseline characteristics. The logarithmic odds of disease progression were modelled to investigate time trends. Random-effects models were used to account for between-study variability; all investigated models included trial duration as a predictor to correct for unequal study durations. Meta-regressions were conducted to assess the prognostic value of a number of study-level baseline variables.

RESULTS:The systematic literature search identified 39 studies, including a total of 19,714 patients. The proportion of patients in placebo controls experiencing a disease progression decreased over the years (p<0.001). Meta-regression identified associated covariates including the size of the study and its duration that in part explained the time trend. Progression probabilities tended to be lower in the second year of a study compared to the first year with a reduction of 28% in progression odds from year 1 to year 2 (p = 0.017).

CONCLUSION:EDSS disease progression exhibits similar behaviour over time as the ARR and point to changes in trial characteristics over the years. This needs to be considered in comparisons between historical and recent trials.

Figure: The fractions of patients with progressing EDSS status over the years.The chances of progression also depend on the study duration (you can see that shorter studies have smaller fractions) but even after accounting for the duration, the decreasing trend remains statistically significant (p<0.0001). The red line shows the estimated regression line for a trial duration of 1 year.

This study demonstrates that the rate of disease progression in the placebo arm (i.e. the arm receiving no active drug) has decreased over time in clinical trials (see above figure).

Why does this matter? One of the anonymous comments on 'Unrelated blogger comments' for September was why a head to head comparison of two drugs needs to be done? Why not simply compare current efficacy figures to those taken from previous trials. The answer is that comparing an old drug/treatment to a newly developed drug in order to demonstrate potency of one drug over another would be erroneous as the relative risk reductions of drug to placebo is different in each study; and if you believe this study even harder now to prove that your drug is effective than thirty years ago!

Part of the problem is the reduction in duration of clinical trials (i.e. less time over which to detect a significant change), and an increase in the size of clinical trials which dilutes out the figures. It is well known in Cancer Research that mortality (i.e whether you are dead or alive) is the defining clinical outcome of efficacy for chemotherapy, whereas disease spread and even recurrence can be marred by simple randomness. It is also why in MS we had to wait 21 years to see whether Betaferon works, because this is the length of time required to demonstrate that there is improved mortality with Betaferon compared to placebo.

They authors also found, the chance of progression is less in the second year of a clinical trial than the first year. This is a recognised entity in the land of statistics, also termed regression to the mean (RTM, see figure below). It occurs when repeated measures are done in the same individual over time, and in general when high (or conversely low values) are observed at one time point they are likely to be followed by less extreme ones nearer to the subjects true average/mean reading.

Figure: regression to the mean.  The first panel shows a Normal distribution of observations for the same subject. The true mean for this subject (shown here as 50 mg/dl) is unknown in practice and we assume it remains constant over time. We assume that the variation is only due to random error (e.g. fluctuations in the HDL cholesterol measurements, or the subject's diet). In the second panel we show an observed HDL cholesterol value (from this Normal distribution) of 30 mg/dl, a relatively low reading for this subject. If we were to observe another value in the same subject it would more likely be >30 mg/dl than <30 mg/dl (third panel). That is, the next observed value would probably be closer to the mean of 50 mg/dl (third panel).

If we take relapsing-remitting MS trials as an example, which use the occurrence of 'x' number of relapses for inclusion into the trial. These relapses can and do settle down spontaneously leading to an apparent improvement in relapse rates. As far as progression is concerned using a shorter time to confirmed disability progression in short study will be more susceptible to the impact of relapses on disability than measuring progression over a longer period.

The moral of this story is design your trial to answer the question you want answered and beware the statistics. 


  1. The plot on the first Figure is misleading: the regression line is drawn for "short" (1-year) studies, but the first actual datapoint is from ca. 1995. The slope of the regression line from 1995 to 2014 is rather small, and then the most visually impressive part of the curve is pure extrapolation. Change the regression model - and a different extrapolated regression line will be obtained. So - yes, beware the statistics

    1. Ignoring the regression line, you'll agree that the rate of progression in studies is declining and barring two small studies majority of the large studies are at populating the top section of the graph?

    2. I don't know, if I only look at green diamonds, I can easily draw a horizontal line at ca. 15-18%, which will go between experimental datapoints (I admit that I can't estimate rms deviations by just looking). I fail to see how 95% confidence interval does not include roughly a half of the data. I agree that the longer studies seem to show a downward trend, but the spread of data is huge.

      Still, thank you for posting and explaining the article, it raises important questions and concerns.


Please note that all comments are moderated and any personal or marketing-related submissions will not be shown.