Tuesday, 25 November 2014

ClinicSpeak: comparing the efficacy of the oral drugs

Can we use modeling to do virtual head-2-head studies; the relative efficacy of the orals. #MSBlog #MSResearch #ClinicSpeak

"It is becoming increasingly important to demonstrate efficacy, or relative efficacy, of one DMT over another so that the cost-effectiveness of the agents can be assessed and the correct level of reimbursement set. Some European countries have made this a priority. One particular country has asked for data on the relative efficacy of fingolimod against the other oral drugs on the market. Short of doing head-2-head studies the next best thing is to model the drugs against each other using published data. How do you do this? The most common method is simply to compare the phase 3 trial outcome data. This is fraught with problems because of the different type of MSers recruited into these studies and the different time epochs these studies were performed in. In addition, the relapse rates, or events, in the comparator or placebo arms differ, making comparison between relative efficacy rates difficult. The best way to tackle this is statistically is using modeling. I was somewhat surprised that there are well-developed methods in statistics for doing this. Why hadn’t the field of MS embraced these methods earlier? My interest in all things MS got me invited onto a panel to try modeling the outcome and made me realise why the 10,000 hour rule is so important. The statisticians who worked on this project are simply amazing and I want to take this opportunity to thank Niklas Bergvall, Richard Nixon, Nikolaos Sfikas and Gary Cutter for opening my eyes to a new world of statistics that the non-expert like me can only dream of being able to do. Statisticians are seriously under-appreciated; not anymore in my books. Although the methods are very complicated and it took me a long time to grasp, they are a major improvement on the comparative methods we have been using to date to compare trial outcomes.”

“I have tried to explain the principles of the modeling method using the embedded slideshow. Most of the problems with across trial comparisons relate to differences in the study population. What we did is simply take all the fingolimod trial data and extract from it subpopulations of MSers that match the baseline characteristics of the MSers studied in the Teriflunomide and DMF studies. When asked how these matched sub-populations from the fingolimod trials did in achieving NEDA (no evident disease activity) and compared them to the published data from the teriflunomide and DMF trials. To do this analysis you need all the raw trial data from the fingolimod trials to interrogate and you need the published data from the teriflunomide and DMF studies. You then define characteristics in the study population that may affect the outcome; these are referred to in stats speak as covariates. For the analyses we used 8 covariates: 
  1. Age 
  2. Gender
  3. Previous DMT use 
  4. Duration of MS 
  5. Number of relapses in the past year 
  6. EDSS score at baseline 
  7. Number of Gd-enhancing T1 lesions at baseline
  8. Cube root of the total volume of T2 lesions
The modeling method uses a conservative approach in that it penalises our assumptions to keep things as simple as possible; it does this by trimming the confidence intervals of the relative risks attributed to each covariate. By doing this you limit the models complexity and only keep the covariates that affect outcome. For the non-statisticians reading this post the so called penalisation factor is called the Akaike information criterion or AIC. We then assessed the model for its goodness-of-fit using another set of statistical rules before accepting it. The message I want to get across is that this assessment is no trivial task and the results are about as close as you can get to a formal head-2-head study from the data we had in hand. In other words an in silico head-2-head study."

"Why NEDA? It is becoming increasingly clear that NEDA, or a form of minimal evidence of disease activity (MEDA), is becoming the treatment target among the vast majority of MSologists, with some well-publicised exceptions (e.g. the United Kingdom). Therefore it made sense to us this outcome as the results may help guide clinical practice. Interestingly, I second guessed these results based on the primary outcome data of the clinical trials, but it is nice to see that the model predicted my assumptions."

"There are limitations to our modeling approach that are based on our assumptions. For example, we assumed that the outcomes of the trials were influenced by the set of 8 covariates above; it is likely, that the results could be affected by additional variables not included in the models, such as the environment at the time these studies were conducted and/or the neurological practice in countries participating in the studies. Unfortunately, we can only adjust for known variables and could not account for subtle unmeasured selection criteria as sources of influence or bias. The other issue is that the way NEDA is reported in these studies is using the baseline or month zero scan. We now rebaseline MSers on DMTs so this analysis will need to be redone when we can access year 2 data only; i.e. NEDA rates after rebaselining at month 12. The current methods put teriflunomide at a disadvantage as it probably takes the longest to start working and it had the most active study subjects based on the event rate in the placebo group. Interestingly, teriflunomide is the only oral DMT to hit significant disability outcomes in both studies, which is why it is level pegging with DMF in our analysis relative to fingolimod. It would be interesting for Genzyme-Sanofi and Biogen-Idec to repeat the same exercise as us using their own data sets; i.e. to triangulate the results. Wouldn't it be interesting if their results were the same or even differed? I would like to challenge them to do the same." 

"Will these results affect clinical practice? They are unlikely to in the UK or Europe, where fingolimod has a 2nd-line license. In the UK you can only use fingolimod after you have failed a 1st-line injectable therapy. I suspect in countries where fingolimod has a 1st-line license this may affect MSer and MSologist choice."

"As far as MSer-choice goes efficacy is only one factors in the decision-making process. I always tell patients that it is ‘horses for courses’; gone are the days when we simply prescribe a drug and leave you on it for years. We now actively monitor response, or non-response, and if you have breakthrough disease we change your therapy. Therefore, I think much bigger differentiators for individuals amongst the oral therapies will be pregnancy, tolerability, side-effects, adherence and safety issues. As I say this I am also acutely aware that fingolimod will be the first small molecule oral medication that comes off patent, which should pave the way for a cheap generic DMT for MSers. This in my opinion will be the most disruptive factor to face the MS DMT market. Any guesses on whether or not the EMA will change fingolimod’s license? Let’s hope so for the sake of MSers. I predict fingolimod will become a 1st-line treatment in Europe long before 2019 in anticipation of it coming off patent. As always economics is the trump card; if it wasn’t we wouldn’t have to resort to complex statistical modeling and posts of this nature.”

"Another disruptor is cladribine; we didn't include it in this analysis as it does not have a license as a DMT in MS. However, it has very good NEDA data and I suspect it would as efficacious, if not more efficacious, than fingolimod. This makes my recent post on the off-label use of cladribine in resource poor countries very pertinent to this post. I really wish we could resuscitate cladribine and get it a licensed as an alternative option for the treatment of RRMS. Oral cladribine has so many positive attributes; it is orally administered as short courses, it is an induction therapy, it ideal treatment for woman wanting to start, or extend their families, side effects are low, it is well tolerated, adherence is really not an issue and there is no secondary autoimmunity. Even the short-term malignancy scare appears to have disappeared although it is likely that long-term risks will remain an issue. The big issues are infection risks and persistent lymphopaenia; these are really problems across many DMTs and we know how to handle them. I hope Merck-Serono or a White Knight is reading this post."

“When reading and assimilating information from this post, please note my conflicts of interest below. I may be biased, but the data is what it is.” 

Epub: Nixon et al. No Evidence of Disease Activity: Indirect Comparisons of Oral Therapies for the Treatment of Relapsing-Remitting Multiple Sclerosis. Adv Ther. 2014 Nov 21. 

INTRODUCTION: No head-to-head trials have compared the efficacy of the oral therapies, fingolimod, dimethyl fumarate and teriflunomide, in multiple sclerosis. Statistical modeling approaches, which control for differences in patient characteristics, can improve indirect comparisons of the efficacy of these therapies.

METHODS: No evidence of disease activity (NEDA) was evaluated as the proportion of MSers free from relapses and 3-month confirmed disability progression (clinical composite), free from gadolinium-enhancing T1 lesions and new or newly enlarged T2 lesions (magnetic resonance imaging composite), or free from all disease measures (overall composite). For each measure, the efficacy of fingolimod was estimated by analyzing individual patient data from fingolimod phase 3 trials using methodologies from studies of other oral therapies. These data were then used to build binomial regression models, which adjusted for differences in baseline characteristics between the studies. Models predicted the indirect relative risk of achieving NEDA status for fingolimod versus dimethyl fumarate or teriflunomide in an average patient from their respective phase 3 trials.

RESULTS: The estimated relative risks of achieving NEDA status for fingolimod versus placebo in a pooled fingolimod trial population were numerically greater (i.e., fingolimod more efficacious) than the estimated relative risks for dimethyl fumarate or teriflunomide versus placebo in each respective trial population. In indirect comparisons, the predicted relative risks for all composite measures were better for fingolimod than comparator when tested against the trial populations of those treated with dimethyl fumarate (relative risk, clinical: 1.21 [95% confidence interval 1.06-1.39]; overall: 1.67 [1.08-2.57]), teriflunomide 7 mg (clinical: 1.22 [1.02-1.46]; overall: 2.01 [1.38-2.93]) and teriflunomide 14 mg (clinical: 1.14 [0.96-1.36]; overall: 1.61 [1.12-2.31]).

CONCLUSION: Our modeling approach suggests that fingolimod therapy results in a higher probability of NEDA than dimethyl fumarate and teriflunomide therapy when phase 3 trial data are indirectly compared and differences between trials are adjusted for.

CoI: multiple. I am a co-author on this paper; it came out of a project presented at the Global Fingolimod Advisory Board (GFAB); I am a sitting member on this board. 


  1. Part of me regrets not starting at Tysabri when I was offered it first-line (with the condition I'm JC-) , how does Tysabri compare to Gilenya in achieving NEDA ?

  2. what's stopping you doing this for cladribine? Klausy should talk to these boffins

    1. Oh if only you knew......................................

    2. i take it klausy is already doing it!?

    3. Nothing is stopping us doing cladribine. However, the analysis had to be done for a particular country's regulators and they did not ask for a comparison with cladribine. We also had to make the call that the paper would be unwieldy with another agent. i can ask them to run the analysis; however, time is money and they are always working on something so they may not be allocated time for this work.

    4. why is it 'unwieldy'? this is an important question, surely. maybe arie can do it?

    5. It has already been done! Not - as far as published data go - for NEDA, but for relapse rate & deteriorating disability:


      Indirect comparisons of oral fingolimod versus natalizumab and cladribine for the treatment of relapsing–remitting multiple sclerosis based on data from FREEDOMS, AFFIRM and CLARITY
      R. Nixon, B. Eckert, G. Cutter, F. Mercier, G. Francis, L. Kappos (Basel, CH; East Hanover, Birmingham, Cambridge, US)

      Results reported suggest there is little to choose between Fingolimod, Cladribine and Natalizumab in terms of relapse rate, whilst Natalizumab seemed to have slight advantage over the other two in preventing disability.

      As of today the study has not been published, perhaps not surprising given significant conflicts of interest involved.

  3. ProfG
    Based on your presentation it suggests that fingolimod is 1.7X better than Tecfidera and 1.7X better than Aubagio. So does this mean that Aubagio is just as good asTecfidera? However, it is being claimed that Tecfidera is much better than Aubagio. Is this correct or do you just think this is selective marketing buy the manufacturers of Tecfidera?

  4. Perhaps a more interesting problem to address with modern statistics tools would be to identify clusters of responders/non-responders in the trial data. Is the apparent difference in the efficacy of fingolimod vs. dimethylfumarate vs. teriflunomide due to greater fraction of responders, ore higher efficacy in "responder sub-population", or higher efficacy across the board?


Please note that all comments are moderated and any personal or marketing-related submissions will not be shown.