Note from the National Guideline Clearinghouse (NGC): The National Institute for Health and Clinical Excellence (NICE) commissioned an independent academic centre to perform a systematic literature review on the technology considered in this appraisal and prepare an assessment report. The Evidence Review Group (ERG) report for this appraisal was prepared by the Centre for Review and Dissemination (CRD) and Centre for Health Economics (CHE) Technology Assessment Group (see the "Availability of Companion Documents" field).
Description and Critique of Manufacturers Approach to Validity Assessment
The ERG's validity assessment of the two studies (TRANSFORMS and FREEDOMS) used by the manufacturer for direct comparisons is shown in Table 3 of the ERG report (see the "Availability of Companion Documents" field).
The studies included in the mixed treatment comparison (MTC) were appraised (in manufacturer's submission) using the criteria of randomisation methods and blinding. Whilst these deal with some basic aspects of trial validity they do not consider allocation concealment, use of an intention-to-treat analysis, selective outcome reporting or the use of a power calculation. The results of the appraisal indicated that most trials were of reasonable quality, based on the criteria assessed.
The ERG did not replicate the validity assessment of the trials in the MTC; since the MTC is not used to inform the economic model this was not considered necessary.
Describe and Critique the Statistical Approach Used
The two trials of head-to head comparisons (TRANSFORMS and FREEDOMS) appraised fingolimod 0.5 mg and 1.25 mg compared, respectively to Avonex 30 µg and to placebo. Consequently there was no attempt statistically to combine the efficacy data from these trials in the clinical effectiveness sections of the submission; this was clearly appropriate.
The fingolimod 0.5 mg arms were combined in the analysis of safety outcomes for comparison with data from the Avonex arm of the TRANSFORMS trial and the placebo arm of the FREEDOMS trial respectively. This appeared reasonable despite the differences between the trials.
Whilst the head-to-head comparisons reported hazard ratios (HRs) for efficacy outcomes, the MTC employed relative risks (RRs). This is potentially problematic as, unlike an odds ratio, the RR is not symmetric. This fact has been demonstrated to be capable of generating anomalous results in an indirect comparison, including for an analysis comparing natalizumab with interferon therapy for the outcome of progression. A further point to note is that the MTC was conducted using PROC GLIMMIX in SAS; this is known to incorporate lower levels of uncertainty around the means than WINBUGS. Therefore it is possible that the analysis may not reflect the full extent of the heterogeneity which was apparent between the trials.
Critique of Submitted Evidence Syntheses
The only synthesis of the two head-to-head trials consisted of a pooling of the fingolimod 0.5 mg arms for the assessment of adverse events relative to the placebo arm of FREEDOMS and the Avonex arm of TRANSFORMS.
An MTC of 18 trials attempted to provide evidence of fingolimod's efficacy on key outcomes of annualised relapse rate (ARR), disability progression and treatment discontinuation due to adverse events. These included trials that had populations who met criteria for relapsing–remitting multiple sclerosis (RRMS) but not necessarily for any of the populations defined by Committee for Medicinal Products for Human Use (CHMP). The submission did not attempt the post-hoc identification of subgroups within these trials. There was also, as the submission noted, very considerable clinical heterogeneity between the trials with respect to permitted and actual prior use of disease-modifying therapy (DMT), duration, and criteria used to define disability progression. An exploration of covariates did not indicate any variables with consistent statistically significant effects on all treatment endpoints. However, two covariates with statistical significance were identified for each of ARR and disability progression.
As a consequence of the heterogeneity, and the fact that it was based on general RRMS populations, the MTC was not subsequently used to inform the economic model.
See Section 4 of the ERG report (see the "Availability of Companion Documents" field) for more information.
The cost-effectiveness evaluation used a decision model, designed as a Markov model, to model disease progression using 21 health states representing different degrees of disease severity and death. Disability progression and conversion to secondary progressive multiple sclerosis (SPMS) were assumed irreversible. The model also accounted for relapses, adverse events, withdrawal and death. Although the occurrence of relapses did not influence the way in which progression was modelled to occur, relapse was modelled to depend on expanded disability status scale (EDSS) score. After withdrawing from fingolimod or Avonex patients were assumed to receive best supportive care (BSC).
The perspective of the analysis of costs was that of the National Health Service (NHS) and Personal Social Services (PSS). Costs were separated into disease costs, administration and monitoring costs and drug acquisition costs. Quality-adjusted life years (QALYs) were used as the measure of outcomes. Both patient and caregiver utility were accounted for and varied by disease severity. Utility adjustments were also applied to account for relapses and adverse events. Treatment with fingolimod or Avonex was assumed to be provided only to RRMS patients with an EDSS score of between 0.0 and 6.0. Patients were modelled to continue to receive these treatments until the treatment was either withdrawn (due to adverse events, disease progression to an EDSS score of above 6 or conversion to SPMS) or a patient died. A 50 year time horizon was used in the model to 'sufficiently capture differences in costs and outcomes'; previous appraisals in MS assessed by NICE adopted time-horizons lower than or equal to 20 years (TA 32 and TA 127). Both costs and benefits were discounted at 3.5%.
Both deterministic and probabilistic sensitivity analysis were carried out by the manufacturer to demonstrate the level of uncertainty around the model results. Despite the non-linear nature of the model, only the deterministic results were presented in the manufacturer's submission.
Summary and Critique of Manufacturer's Submitted Economic Evaluation by the ERG
An overall summary of the manufacturer's approach and signposts to the relevant sections in the manufacturer's submission are reported in Table 7 of the ERG report (see the "Availability of Companion Documents" field).
Table 8 of the ERG report summarises the economic submission using a checklist based on NICE's reference case and other methodological recommendations, and the ERG's comments on whether the de-novo evaluation meets the requirements of these recommendations.
Model validation was dealt with by the manufacturer in a limited manner and seems to have been restricted to checking that formulas in spreadsheets were typed correctly and ensuring referenced values were copied into the model correctly. Some extreme value testing of the model was carried out to ensure that the model behaved as it was described to in the submission (e.g., setting mortality rate to 0 and observing that the model predicts no deaths).
No internal validation or goodness of fit was reported comparing the model predictions with observations from the FREEDOMS, TRANSFORMS or Ontario studies, the three main sources of evidence used to inform the model. No external validation was reported comparing the model predicted results with other trials or other published model results. Clinical expertise was not utilised to verify that the model captured a plausible abstraction of the disease or made clinically plausible predictions.
The ERG has attempted to check the model predictions against the results observed in the trials throughout the current section and have consistently found divergences between them.
See Sections 5 and 6 of the ERG report (see the "Availability of Companion Documents" field) for additional information on cost-effectiveness analysis.