Note from the National Guideline Clearinghouse (NGC): The National Institute for Health and Clinical Excellence (NICE) commissioned an independent academic centre to perform a systematic literature review on the technology considered in this appraisal and prepare an assessment report. The Assessment Group (AG) report for this technology appraisal was prepared by Liverpool Reviews and Implementation Group (LRiG), University of Liverpool (see the "Availability of Companion Documents" field).
Data Abstraction Strategy
Data relating to both study design and quality were extracted by one reviewer into a Microsoft Access® database and were cross checked by a second reviewer. Where multiple publications of the same study were identified, data were extracted and reported as a single study.
Critical Appraisal Strategy
The quality of the included clinical-effectiveness studies was assessed by one reviewer and checked by a second reviewer according to criteria based on Centre for Reviews and Dissemination (CRD) Report 4. The checklist used to critically appraise the included studies is specific to randomised controlled trials (RCTs); for the non-RCT studies a modified version of this checklist was used. All relevant information was tabulated and summarised within the text of the report. Full details and results of the quality assessment strategy for clinical effectiveness studies are reported in Appendix 4 of the Assessment Report (see the "Availability of Companion Documents"' field).
Methods of Data Synthesis
Results of the data extraction are summarised in structured tables and as a narrative description. A standard meta-analysis was planned if sufficient clinically and statistically homogeneous data were available from the included studies. The primary outcomes identified for the evidence synthesis were systemic reaction to field sting (FS) or sting challenge (SC) during treatment and/or adverse reactions to venom immunotherapy (VIT). Secondary outcomes included local reaction to VIT, local reaction to FS or SC, number of stings, deaths.
The Assessment Group planned to extract number of events for each outcome and total number of people in each treatment arm in order to calculate odds ratios (OR) and the correspondent 95% confidence intervals for each study. Studies with no events in both arms would be excluded from analysis. All analyses were planned based on the intention to treat (ITT) population where possible. Where appropriate, the levels of clinical and methodological heterogeneity would be investigated, and statistical heterogeneity would be assessed using Q- and I2-statistics. Given the small number of trials available, a fixed-effects model was planned using the 'metan' command within STATA Version 9.2 where pooling was appropriate.
If the data allowed, a mixed treatment comparison (MTC) of relevant comparators to venom immunotherapy using Pharmalgen products (PhVIT) would be considered. This approach fulfils the objective of providing simultaneous comparison of all the relevant treatment alternatives, and can provide information about the associated decision uncertainty or sufficient information for economic evaluation. Hence, for the purposes of decision-making, a Bayesian MTC framework would be adopted to synthesise information on all technologies simultaneously using Markov Chain-Monte Carlo (MCMC) methods to estimate the posterior distributions for the outcomes of interest.
Indirect Analysis and Mixed Treatment Comparisons
The possibility of conducting a MTC was investigated when no head-to-head studies were identified that compared PhVIT and alternative treatment options available in the National Health Service (NHS) without VIT such as: advice on the avoidance of bee and wasp venom; high-dose antihistamines (HDA); adrenaline auto-injector (AAI) prescription and training. However, given the small number of trials and lack of head-to-head comparisons of PhVIT versus any intervention, pooling of all outcomes using standard meta-analysis was not possible. Any indirect analysis comparing PhVIT with any other intervention (including different doses and administration protocols of PhVIT) would be inappropriate owing to sparse data, heterogeneity in the study designs and in the characteristics of non-PhVIT and non-VIT interventions.
See Section 5 of the Assessment Report (see the "Availability of Companion Documents" field) for additional information.
The manufacturer of PhVIT did not submit any clinical or cost-effectiveness evidence to NICE. The AG developed a de novo economic model designed specifically to compare the cost-effectiveness of PhVIT with currently available NHS interventions in people with a history of type 1 immunoglobulin E (IgE) mediated systemic allergic reactions to bee and wasp venom.
An overview of the AG's de novo economic model is summarised in Table 17 of the Assessment Report (see the "Availability of Companion Documents" field).
The economic model was constructed as a 1 year cohort decision tree that was extrapolated to have a horizon of multiple years with the only changes being a reduction in the size of the cohort at the end of each year due to sting related death or death from other causes. The average age of the cohort increased with the time horizon of the model with all cause mortality rates changing as the average age of the cohort increased. Development of a Markov model was not appropriate for disease modelling of the decision problem. To illustrate, with the exception of death, there was no transition into a state that results in changes to the key parameters; for example, being stung did not change the probability of experiencing a systemic reaction from future stings.
The available evidence for the key pathway parameters (likelihood of sting, resulting systemic reaction under different treatment arms and the likelihood of death following systemic reaction) was weak. As such, construction of probability distributions around these parameters was not feasible. Instead, a deterministic model was produced using the best available estimates with sensitivity and scenario analyses employed to test the impact of changing the parameters within plausible ranges.
A schematic of the first year of the model for PhVIT + AAI + HDA is shown in Figure 2 of the Assessment Report (see the "Availability of Companion Documents" field). The schematic for subsequent years was identical with the exception that the updosing phase of VIT was no longer present and after PhVIT had stopped the maintenance phase ended. The model then simplified into the number of stings per patient per year with resulting systemic reactions and the number of deaths from other causes. For the other treatment arms the model was essentially this simplified version of the intervention arm. The cohort was defined as 1000 patients who receive a full course of PhVIT; any extra costs due to non-adherence to treatment were considered implicitly if maintenance continued for 5 years rather than 3 years as described in the sensitivity analysis.
Internal Validation of Assessment Group Model
During model construction the algorithms within the model were checked using extreme value analysis for parameters to ensure that results generated were within acceptable bounds. To verify the accuracy of the model, key algorithms within the model were checked by an independent statistician. On completion, the model was assessed and validated by a team of external economists and statisticians.
External Validation of Assessment Group Model
The model was also cross checked by an external consultant. The economic model was checked for functionality, clarity, accuracy, consistency and validity. Validation of calculated parameters within the model was carried out where possible against observational studies. However given that this is de novo economic model, it was not possible for the external consultant to conduct validation regarding final results.
See Section 6 of the Assessment Report (see the "Availability of Companion Documents" field) for additional information.