Wednesday, 24 August 2016

ClinicSpeak: risk benefit, risk harm analyses

Do we need a new way to communicate benefits and risks? #ClinicSpeak #MSBlog #MSResearch
'Lies, damned lies, and statistics', Mark Twain

"The analysis below looks at the benefit and risk of DMTs from a different statistical perspective, i.e. how many patients do you need to treat with DMT to benefit, or harm, one patient? 

NNTB = number-needed-to-treat to benefit
NNTH = number-needed-to-treat to harm
LHH = likelihood to be helped-or-harmed
LHH = NNTH/NNTB ratio
AAR = annualized relapse rate
PPR-F = proportion of relapse-free patients
PP-F-CDPS3M = proportion of disability progression free patients confirmed or sustained at 3 months

When analysed like this it is remarkable how well subcutaneous interferon-beta-1a (Rebif) does compared to the newer DMTs. Of the newer agents Natalizumab fares the best. Is this a compelling way to present data to people with MS? If I presented data in this way I suspect many of my patients may select interferon-beta as their agent of choice. The problem with this analysis is that it classifies harm very broadly and not all 'harmful events' are necessarily serious or life threatening. For example, raised liver enzyme that are asymptomatic and reversible is very different to PML or a life-threatening infusion reaction. In addition, this type of analysis does not take into account individual patient characteristics (prognostic profile and baseline disease activity) nor does it take into account lifestyle issues (family planning, cosmetic issues, etc.). I may be wrong but I am not sure it will be easy to present the relative risks and benefits of each of these DMTs to patients in this way without getting bogged down in the small print about rare adverse events. However, I may be wrong. What do you think?"


Mendes et al. Benefit-Risk of Therapies for Relapsing-Remitting Multiple Sclerosis: Testing the Number Needed to Treat to Benefit (NNTB), Number Needed to Treat to Harm (NNTH) and the Likelihood to be Helped or Harmed (LHH): A Systematic Review and Meta-Analysis. CNS Drugs. 2016 Aug 18. [Epub]

OBJECTIVE: This study aimed to test the number needed to treat to benefit (NNTB) and to harm (NNTH), and the likelihood to be helped or harmed (LHH) when assessing benefits, risks, and benefit-risk ratios of disease-modifying treatments (DMTs) approved for relapsing-remitting multiple sclerosis (RRMS).

METHODS: In May 2016, we conducted a systematic review using the PubMed and Cochrane Central Register of Controlled Trials databases to identify phase III, randomized controlled trials with a duration of ≥2 years that assessed first-line (dimethyl fumarate [DMF], glatiramer acetate [GA], β-interferons [IFN], and teriflunomide) or second-line (alemtuzumab, fingolimod, and natalizumab) DMTs in patients with RRMS. Meta-analyses were performed to estimate relative risks (RRs) on annualized relapse rate (ARR), proportion of relapse-free patients (PPR-F), disability progression (PP-F-CDPS3M), and safety outcomes. NNTB and NNTH values were calculated applying RRs to control event rates. LHH was calculated as NNTH/NNTB ratio.

RESULTS: The lowest NNTBs on ARR, PPR-F, and PP-F-CDPS3M were found with IFN-β-1a-SC (NNTB 3, 95 % CI 2-4; NNTB 7, 95 % CI 4-18; NNTB 4, 95 % CI 3-7, respectively) and natalizumab (NNTB 2, 95 % CI 2-3; NNTB 4, 95 % CI 3-6; NNTB 9, 95 % CI 6-19, respectively). The lowest NNTH on adverse events leading to treatment discontinuation was found with IFN-β-1b (NNTH 14, 95 % 2-426) versus placebo; a protective effect was noted with alemtuzumab versus IFN-β-1a-SC (NNTB 22, 95 % 17-41). LHHs >1 were more frequent with IFN-β-1a-SC and natalizumab.

CONCLUSIONS: These metrics may be valuable for benefit-risk assessments, as they reflect baseline risks and are easily interpreted. Before making treatment decisions, clinicians must acknowledge that a higher RR reduction with drug A as compared with drug B (versus a common comparator in trial A and trial B, respectively) does not necessarily mean that the number of patients needed to be treated for one patient to encounter one additional outcome of interest over a defined period of time is lower with drug A than with drug B. Overall, IFN-β-1a-SC and natalizumab seem to have the most favourable benefit-risk ratios among first- and second-line DMTs, respectively.

CoI: multiple

26 comments:

  1. The lack of comments on this post indicates to me that the NNT and NNH are academic constructs that don't resonate with MSers. It is becoming increasingly clear that the decision-making responsibility is going full circle with more and more of my patients wanting me to make the call. Is that correct?

    ReplyDelete
    Replies
    1. I don't know how these do or don't resonate with other MS'ers but it would be easier for me to comment if it was clearer about how a neurologist would use these measures in discussion with a newly-diagnosed patient or with a MS'er who was considering changing DMT.

      When I was diagnosed in 2008 there were four available DMTs and the information I was given compared each DMT with a placebo and gave ARR data.

      Now there are many more DMTs and there appears to be greater understanding of factors which indicate a better / poorer prognosis it would be beneficial for people to use a tool to give them an indication of the risks /benefits associated with each drug given their individual prognostic indicator.

      The patient could do this alongside a neurologist / MS specialist nurse or by themselves ahead of a clinic appointment.

      Personally I don't want a neurologist making decisions for me but I do want their expertise to assist me in reaching that decision. I know of people who want their neurologist to make the relevant decision because they're overwhelmed by the diagnosis / are serial ditherers / don't feel they know enough about MS, DMTs etc. The danger, for the patient, of the neurologist making the decision is that the patient doesn't know how up to date the neuro's MS / DMT knowledge is and their attitude to prescribing.

      Delete
  2. % reduction of relapse rates and % of side effects are much easier to understand. You'd have to spend a lot longer explaining those acronyms than my % chance of achieving NEDA with each drug.

    I'd say that the neurologist opinion is important but what if they're not up to date? And surely it's hard to take into account all the lifestyle and risk factors that make up someone's individual circumstances in a busy NHS clinic? I'm not sure I'd want the decision to be taken entirely by someone else - although a strong opinion is good!

    ReplyDelete
  3. This is probably very word salad for many people with MS, Prof G. I have an advanced degree in a science-adjacent field of art and post-Lemtrada have regained the ability to read a novel, but even my eyes started glazing over the acronyms.

    ReplyDelete
  4. I'm in Australia and from my observations, there are patients who want the doctors to make the decision, then there are those patients who want their doctor to provide the information so they can make the decision. The latter kind of doctor seem few and far between sometimes.

    I'm in the latter group and fortunately I have a wonderful neurologist who respects that it is my life and that I'm aware of the risks, in particular the exact likelihood of those risks eventuating.

    I think an important role of the neurologist is to help the patient to understand the available statistics relating to risks and their likelihood, whilst ensuring they don't disregard a very low risk or disproportionately panic about a high risk.

    All things considered, I went with Lemtrada as my neurologist and I felt it was right for me.

    ReplyDelete
  5. Dear ...........

    Please do not be offended but you may notice that some of your posts are not being posted.

    You are obviously taking a lot of time to write huge responses or comments but please be aware we can only see about 150 words and cannot see the whole post.
    We don't want to have to post comments to allow us to read them, only then to have to delete them.

    However on this 150 words we may decide to post or spam (we may not all agree whether to spam or not and past experience may influence this, we may not have the time to read through the posts) but I guess in the new era we may take the conservative approach and not post.

    If comments seemly contain advice, especially when unrelated to the contents of the post, they are likely to spammed. Notably we just spammed your post concerning your experiences/view on LND. We accept that many people use this but there is no approved use, but you appear evangelical which you may not be intending to do. You have an anecdote. We appreciate that you are sharing your experience, but others may lack such experience, and it is not the function of this site to discuss ancedotes and remember you are providing more details about yourself when discussing anecdotes

    Sorry

    Best wishes

    ReplyDelete
    Replies
    1. We do not discuss anecdotes unless they are a published case report:-)

      Delete
  6. Re: "The lack of comments on this post indicates to me that the NNT and NNH are academic constructs that don't resonate with MSers."

    Prof G, there seems to be a sluggishness on the blog ever since you did that post about losing your mojo. The blog was relaunched but with less momentum than before. A lot of the old commentators that made the blog interesting and fun have gone elsewhere or been banned. I visit the blog less often now because it doesn't feel as interesting than before.

    You guys do a great job, though. I just want more variety.

    ReplyDelete
    Replies
    1. So true. It's less lively. I sort of liked the back and forth between Team G and certain nihilistic commentators. It made for very interesting reading and breaking things down. I loved the political posts that incurred the wrath of Dr Dre and then everyone weighing in to prove him wrong. It was great reading material.

      I think this blog should be a place of debate. It should also invite proposals on things we want to discuss concerning MS like OT services and the future of the NHS.

      Delete
    2. We have tried to get more people posting,but you can only lead a horse to water you can't make them drink. What kind of variety do you want?

      Delete
    3. Re: "What kind of variety do you want?"

      Prof G, you have to admit that the blog has incurred a change this summer. I love the science posts. I love the political posts. I also love fellow MSers actively debating posts. The blog seems to be more austere nowadays. It's really bland, but it was''t like this before the "mojo" post. Something has changed and your readership may have suffered as a result. Am I the only one feeling this way?

      The blog feels really unilateral right now. You need to invite discussion and debate issues. Get us thinking.

      Delete
    4. I think that the blog, perhaps for the best reasons, wanted to get rid of disruptive commentators because they were in danger of misguiding MSers. It created great debates, but was frustrating Team G.

      The blog does need some new life, though. Yet I don't know what exactly. I'd like MD2 posting more. I loved MD2 debating with VV and Dre. That was really good. I miss the old Vit D doctor called Gav, too. The blog has to be more than two doctors.

      Delete
    5. It is good to see Dr Dre behaving himself under a new pseudonym.

      Delete
    6. When did Dre get banned? I thought he just stopped posting. His comments were seriously intermittent at the best of times. Dre was a character though.

      Prof G, don't be disheartened by the lack of recent MSer engagement. People are on holiday.

      Delete
    7. I like the clinic posts the most. I gain the most from them anyway. I still really rate this blog. Keep doing what your doing as it is of value.

      Delete
    8. My posts have not been getting published, one of the mouse docs tells me it's because they are "rants".

      Why should I bother commenting if you are now so picky about what you publish? (it's a waste of my time).

      Of course, I still read the blog because I have nothing alternative to read :)

      Delete
    9. Yep rants are often binned, we have been asked not to engage in/with personals slurs, rants, etc, post closure , yes it is no fun but it ProfG would rather this does not happen as is it likely to get the blog-closed down again.

      Rather than take the bait, not engaging is the more likely response.
      and not having the bait in sight is the easiest ways to not engage as it is often red to a bull

      Monologues are at risk also as we cannot see the whole post. ProfG has not got the time to moderate the comments to be consistent. What I think is OK is different from MD2, is different from DrK and is different from ProfG.

      Sometimes stuff goes straight to spam without us doing anything and unless we remove it there it may stay as we do not have the time to be conferring who did what.

      Sorry.

      Delete
    10. No need to be sorry. Thank you for the explanation.

      I'm here for as long as the blog content meets my needs, particularly as there is no practical alternative.

      How the blog is run is up to the administrators. My answer was in response to the following comments:

      AnonymousThursday, August 25, 2016 7:30:00 pm
      So true. It's less lively. I sort of liked the back and forth between Team G and certain nihilistic commentators. It made for very interesting reading and breaking things down. I loved the political posts that incurred the wrath of Dr Dre and then everyone weighing in to prove him wrong. It was great reading material.

      I think this blog should be a place of debate. It should also invite proposals on things we want to discuss concerning MS like OT services and the future of the NHS.


      Gavin GiovannoniFriday, August 26, 2016 1:25:00 am
      We have tried to get more people posting,but you can only lead a horse to water you can't make them drink. What kind of variety do you want?

      Delete
    11. Found this in spam.

      Yes having abit of too and frow could be fun, but we could over step the mark and get us into bother.

      What kind of variety do you want?

      Different people to do posts, meaning that each of us is spending less time on the blog. However it can be a brutal place for the Newbie, for example some of the pHD students were ripped apart for some of the stuff they were involved with....they were wary about doing things again. They have now left...hope the scars are not too bad, but one got teased so much they got us to remove the content

      Delete
  7. Re: "The lack of comments on this post indicates to me that the NNT and NNH are academic constructs that don't resonate with MSers."
    I assume that is:
    NNTB = number-needed-to-treat to benefit
    NNTH = number-needed-to-treat to harm
    Are NNTB and NNTH integer variables? The values in the abstract for those variables are integers. I think I get the concept.

    Nevertheless I do not follow:
    LHH = likelihood to be helped-or-harmed
    LHH = NNTH/NNTB ratio
    LLH seems a ratio of Harmed to Benefited not to do with helped-or-harmed.

    ReplyDelete
    Replies
    1. Lol. I like your answer. It illustrates that the lack of engagement could be from a place of "huh, you want to summarise my life down to a formula?".

      Delete
    2. Re. Lack of engagement...maybe we don't (understand/like/understand the context of e.g. sarcastic, serious, joking) the comment and therefore we do not wish to engage:-(.

      There are many a time when I see red mist but others do not see the insult that is the problem of trying to understand what the writer means it is not always simple.

      Delete
  8. I remember getting into an argument with my neurologist about NNT vs. ARR. It's kind of a patronizing way of framing things. Is it just me, or are the harms on a different order of magnitudes from the helps? Not incurring permanent disability vs. injection site reactions?

    ReplyDelete
  9. Its goes over my head hence not commented. Whats wrong with the MS Decisions tool or something similar?

    ReplyDelete
  10. Re. "It is becoming increasingly clear that the decision-making responsibility is going full circle with more and more of my patients wanting me to make the call. Is that correct?"

    I'd like expert opinion from my neurologist. Guidance/explanation.

    ReplyDelete
  11. This is an important topic that deserves to be debated. Of course, this is just another way/metric for presenting results. Despite physicians and ptients may be more confortable interpreting relative risks and percentages, it should be noted that sometimes relative risk reductions may be measliding. Therefore, ideally, results should be presented as relative and absolute risks (where NNT is included). For example, a relative risk increase in the proportion of patients with a given serious adverse event equals 80% with treatment versus placebo irrespectively of the proportion of patients with serious adverse events being 50% with treatment and 10% with placebo or 0.05% with treatment and 0.01% with placebo. Thus, relative risks may be not enough to get an overall picture of the risk. The absolute risk reduction will tell us that the absolute reduction is of 40% in scenario A and 0.04% in scenario B. Thus, the NNTH would be 2.5 for scenario A and 2500 for scenario B over a defined period of time. I believe that this is very important from a clinical perspective. Both relative risks and absolute risks should be considered when selecting treatments.

    ReplyDelete

Please note that all comments are moderated and any personal or marketing-related submissions will not be shown.