Features, process and methods of early health technology assessment to inform developers of health technologies: a proposed framework and application to diagnostic technologies

Key points • This thesis aimed to develop a framework to help people undertaking health technology assessment of health technologies in development (termed ‘analysts’) approach their work. • It used a literature review and learning from five case studies to develop a framework which described the features of this type of health technology assessment, a generic process for undertaking the work and described which methods might be appropriate. Although the case studies are all diagnostic technologies, the framework would be appropriate for those working with small and medium-sized developers of any type of health technology. • Analysts need to be aware that HTA for developers of health technologies and their investors (termed ‘developers’) is different to HTA undertaken later in the technology’s lifecycle. Compared to HTA undertaken at market access stage, HTA undertaken to inform developers (termed ‘development-focused HTA’ or ‘DF-HTA’) informs a broader range of decisions and has to deal with a greater level of uncertainty, as the precise use of the technology may not yet be known and there is likely to be little evidence specific to the technology. • Although the process of DF-HTA will be familiar to analysts in that it involves the assessment of clinical and economic value, DF-HTA is more iterative in nature and does not have strict methodological guidelines to follow, as is often the case for HTA at market access stage. Analysts can take a more positive stance and put themselves in the shoes of the developer to consider what analysis would be useful to the developer at this stage of development. • The methods would be familiar to those carrying out health technology assessment but need some modification when the work is to inform developers. This is particularly relevant when resources for analysis are constrained, which is often the case when assessing technologies in development, many of which are likely to fail. In practice, this means that methods may be simpler. There is also more emphasis on methods of stakeholder consultation as a result of the lack of clinical evidence. • The framework would benefit from validation through prospective use in further case studies by external groups of analysts. This is particularly important because much HTA work undertaken for commercial developers is not published due to commercial sensitivity. Background and aims The development of health technologies is expensive and high risk, as many technologies in development fail to achieve commercial success. There are many choices to be made during the development process and the decisions made at this time are likely to influence the future success of the technology. It has been suggested that health technology assessment (HTA) undertaken during the development process may help to reduce the risk of failure or to accelerate failure and reduce research waste. The form of HTA used to inform developers about some of the decisions they need to make during the development process has been termed ‘Early HTA’. However, the current ‘Early HTA’ literature includes work which is not intended to directly inform developers (such as HTA accompanying managed access agreements) and much of the HTA that is conducted to inform developers is not published. As a result, there is a lack of clear guidance for the analyst on how to approach HTA to inform developers of medical technologies. This thesis proposes a framework to aid analysts (people undertaking HTA assessments) who are working with developers of medical technologies. The term developers is used throughout to describe the individuals or companies who are developing a medical technology and their investors/funders. Large pharmaceutical and medical device companies tend to have in-house HTA resource (often termed Health Economics and Outcome Research (HEOR) or market access teams). For this reason, the framework is aimed primarily at academic and consultant analysts working with small or medium-sized enterprises (SMEs). The framework comprises three sections: features, process and methods of HTA to inform developers (termed development-focused or DF-HTA). Features means the characteristics of DF-HTA. Process describes the broad activities of DF-HTA. Methods describes the analytical approach to those activities or how the analyst would undertake the activity. The framework was developed alongside five case studies in the assessment of diagnostic technologies which are included to illustrate the framework of features, process and methods. Methods A pearl-growing literature review was undertaken in October 2017 and refreshed in February 2019. Pearl-growing involves searching the citations and references of articles of interest until no further papers are being retrieved. The aim was to retrieve literature which informed an understanding of early health technology assessment and included both methods papers and applied studies. An iterative process was undertaken to develop a framework of DF-HTA including features, process and methods. An initial list of features of DF-HTA was developed then refined and expanded using an iterative process informed by the content of articles identified as being particularly informative and the experience of undertaking the case studies. Frameworks for the process of DF-HTA identified in the literature review were used as the basis for the development of a generic process of DF-HTA. Methods useful for DF-HTA were extracted from review articles. Methods of research and development or other commercial activities were excluded. Terminology was streamlined where similar terms were describing the same method. Five case studies were used to illustrate the framework of DF-HTA. For each of the case studies the features were compared to the list of features and areas of non-conformity identified and discussed. The process of DF-HTA included clinical value assessment in all cases and economic value assessment in four of the five case studies. Methods were selected from the methods identified as useful for DF-HTA. Results A total of 152 early HTA papers were identified of which 88 were judged to be aimed at informing developers. These comprised 56 methods, 61 applied and 35 methods and applied papers in early HTA of which 43 methods, 25 applied and 20 methods and applied were aimed to inform developers. A proposed framework of DF-HTA was developed including the features, a generic process and a range of methods suitable for DF-HTA. Ten features characterising DF-HTA were identified including six which had not been previously identified: audience; timing; decisions informed; available evidence; underlying user objective; decision space; business model; resources for analysis; stance of analysis and burden of proof. The proposed generic process of DF-HTA comprises two core aspects of DF-HTA identified in all the frameworks found in the literature – clinical value assessment and economic value assessment. Clinical value assessment considers what impact the technology might have on clinical practice and ultimately upon health (and wider social) outcomes. Economic value assessment builds on the clinical value assessment to consider the economic impact of changes in healthcare resource use and other economic value drivers such as productivity effects. Eight broad groups of methods useful in DF-HTA were identified from review articles: care pathway analysis; qualitative methods of stakeholder interaction; literature review; multi-criteria decision analysis; discrete choice experiments; expert opinion and elicitation; cost effectiveness analysis and value of information analysis. The framework was illustrated using the case studies. The features of DF-HTA were evident in three case studies of technologies in development but not in two case studies concerning technologies already in clinical use. The process and methods used in the case studies aligned with the generic process of DF-HTA. All the case studies began with a clinical value assessment using decision models to map the existing and potential clinical pathways. For the three DF-HTA case studies parameter estimates came from the literature and expert opinion initially supplemented by evidence from small clinical studies specific to the technology where this was available. The case studies show that an initial clinical model is able to distinguish technologies with potential value from those with little potential value. Cost-effectiveness or cost consequence models were able to identify potential economic value, indicate design factors which required clarification before further development and identify parameters likely to have a significant impact on potential economic value. The two case studies which involved technologies which were already in clinical use were able to show, using similar methods to DF-HTA and in situations where evidence was limited, cost savings and consequences of expanding a testing programme and introducing a triage test. These case studies demonstrate the need for alternatives to cost-effectiveness using methods appropriate at market access/adoption stage when resources are limited. Conclusion The main contribution of the thesis was to propose a framework of features, process and methods of DF-HTA. A secondary contribution was to provide five empirical examples, three in DF-HTA and two in early economic evaluation of diagnostic technologies. The case studies use simple models that can be readily used to provide an indication of the potential value of a technology whilst it is under development or during some form of expedited review. The main limitation concerns the likely incompleteness and unrepresentativeness of HTA studies to inform developers in the academic literature. This is because of commercial confidentiality and a lack of an incentive to publish. This thesis suggests that developers should be encouraged to consult an HTA practitioner at an early stage in the development. Where development is technology-driven (i.e. the technology is developed without a precise clinical need identified), DF-HTA can help developers to ‘position’ their technology, articulate value propositions and engage with stakeholders. This process can sometimes change the design or intended direction of the technology in development and can inform evidence generation strategy. For analysts, the framework should provide guidance on the nature, process and methods of HTA to meet the needs of the developer. For policy-makers the thesis suggests that it is possible and desirable to encourage the articulation of a value proposition (particularly for translational research) within funding applications. It is also essential to support translational research bodies which facilitate links between commercial entities, academic and clinical researchers, clinicians, regulators and reimbursement agencies. Policy-makers may also wish to fund the development of full disease models which could be used to rapidly and robustly evaluate any proposed technology (in technology-driven development) as well as determining areas of greatest need to inform specific calls for innovation (needs-driven development). Further research exploring the features of in-house and unpublished DF-HTA and the usefulness of this framework of DF-HTA to developers and analysts would be valuable. The decision-making process for the adoption of many technologies is not clear and transparent and research into the evidence relied upon by decision-makers in different settings would be useful. The extent to which full disease models have been or could be used to assess the value proposition for innovative technologies would also be a useful area of research

Evaluation of triage tests when existing test capacity is constrained: application to rapid diagnostic testing in COVID-19

Objectives: A triage test is used to determine which patients will undergo an existing or ‘reference’ test. This paper explores the potential value of triage tests before reference tests when capacity of the reference test is constrained. Methods: We developed a simple model with inputs: prevalence, sensitivity, specificity and reference test capacity. We included a case study of rapid diagnostic tests for recombinant SARS-CoV- 2antigens used as triage tests before a reference polymerase chain reaction test. Performance data is taken from evaluation by the Foundation for Innovative New Diagnostics. Results: When reference test capacity is constrained, the use of a triage test leads to a relative expansion of the population tested and cases identified, both are higher with a high specificity test. When reference test capacity is not constrained, the introduction of a triage test can be assessed using a standard cost-consequence or cost-utility framework balancing the benefit of the reduction in the number of reference tests required against the disbenefit of missed cases. In this case a test with high specificity leads to the greatest reduction in the number of reference tests required and to the greatest number of missed cases. In the constrained case, the advantage of a triage testing strategy in terms of population covered and cases identified reduces as prevalence increases. In the unconstrained case, the reduction in reference tests required is reduced and cases missed increase as prevalence rises. Conclusion: When availability of reference test is constrained, tests added in a triage position do not need high levels of accuracy to increase number of cases diagnosed. This has implications in many disease areas including COVID-19

A different animal? Identifying the features of health technology assessment for developers of medical technologies

Health technology assessment (HTA) conducted to inform developers of health technologies (development-focused HTA, DF-HTA) has a number of distinct features when compared to HTA conducted to inform usage decisions (use-focused HTA). To conduct effective DF-HTA, it is important that analysts are aware of its distinct features as analyses are often not published. We set out a framework of ten features, drawn from the literature and our own experience: a target audience of developers and investors; an underlying user objective to maximize return on investment; a broad range of decisions to inform; wide decision space; reduced evidence available; earlier timing of analysis; fluid business model; constrained resources for analysis; a positive stance of analysis; and a “consumer”-specific burden of proof. This paper presents a framework of ten features of DF-HTA intended to initiate debate as well as provide an introduction for analysts unfamiliar with the field

Synthetic control methodology as a tool for evaluating population-level health interventions

Background: Many public health interventions cannot be evaluated using randomised controlled trials so they rely on the assessment of observational data. Techniques for evaluating public health interventions using observational data include interrupted time series analysis, panel data regression-based approaches, regression discontinuity and instrumental variable approaches. The inclusion of a counterfactual improves causal inference for approaches based on time series analysis, but the selection of a suitable counterfactual or control area can be problematic. The synthetic control method builds a counterfactual using a weighted combination of potential control units. Methods: We explain the synthetic control method, summarise its use in health research to date, set out its advantages, assumptions and limitations and describe its implementation through a case study of life expectancy following German reunification. Results: Advantages of the synthetic control method are that it offers an approach suitable when there is a small number of treated units and control units and it does not rely on parallel preimplementation trends like difference in difference methods. The credibility of the result relies on achieving a good preimplementation fit for the outcome of interest between treated unit and synthetic control. If a good preimplementation fit is established over an extended period of time, a discrepancy in the outcome variable following the intervention can be interpreted as an intervention effect. It is critical that the synthetic control is built from a pool of potential controls that are similar to the treated unit. There is currently no consensus on what constitutes a ‘good fit’ or how to judge similarity. Traditional statistical inference is not appropriate with this approach, although alternatives are available. From our review, we noted that the synthetic control method has been underused in public health. Conclusions: Synthetic control methods are a valuable addition to the range of approaches for evaluating public health interventions when randomisation is impractical. They deserve to be more widely applied, ideally in combination with other methods so that the dependence of findings on particular assumptions can be assessed

Do age, period or cohort effects explain circulatory disease mortality trends, Scotland 1974-2015?

Objective: We aimed to explore whether age, period or cohort effects explain the trends and inequalities in ischaemic heart disease (IHD) and cerebrovascular disease (CeVD) mortality in Scotland. Methods: We analysed IHD and CeVD deaths for 1974–2015 by sex, age and area deprivation, visually explored the data using heatmaps and dotplots and built regression models. Results: CeVD mortality improved steadily over time while IHD mortality improved more rapidly from the late 1980s. Age effects were evident; both outcomes showed an exponential relationship with age for all except males for IHD in the 1980s and 1990s. The mortality profiles by age became older, although improvement was slower for those aged <50 years for IHD, especially for males, and faster for CeVD in females aged <65 years. Rates were higher, and inequalities greater, among males, especially for IHD. For IHD, increased risk for males over females reduced with age (incidence rate ratio for 41–50 year old males=4.28 (95% CI 4.12 to 4.44) and 1.17 (95% CI 1.16 to 1.18) for 71–80 year olds). Inequalities in IHD mortality by area deprivation persisted over time, increasing from around 10% to around 25% higher risk in the most deprived areas between 1974 and 1986 before declining in absolute terms from around 2000. Inequalities for CeVD increased after the late 1980s. Conclusions: IHD and CeVD mortality in Scotland exhibit age but not recent distinct period or cohort effects. The improvements in mortality rates have been more sustained for CeVD and inequalities greater for IHD

Context-specific economic evaluation for molecular pathology tests: An application in colorectal cancer in the West of Scotland.

The cost-effectiveness of molecular pathology testing is highly context dependent. The field is fast-moving, and national health technology assessment may not be relevant or timely for local decision makers. This study illustrates a method of context-specific economic evaluation that can be carried out in a limited timescale without extensive resources. We established a multi-disciplinary group including an oncologist, pathologists and a health economist. We set out diagnostic and treatment pathways and costs using registry data, health technology assessments, guidelines, audit data, and estimates from the group. Sensitivity analysis varied input parameters across plausible ranges. The evaluation setting was the West of Scotland and UK NHS perspective was adopted. The evaluation was assessed against the AdHopHTA checklist for hospital-based health technology assessment. A context-specific economic evaluation could be carried out on a timely basis using limited resources. The evaluation met all relevant criteria in the AdHopHTA checklist. Health outcomes were expected to be at least equal to the current strategy. Annual cost savings of £637,000 were estimated resulting primarily from a reduction in the proportion of patients receiving intravenous infusional chemotherapy regimens. The result was not sensitive to any parameter. The data driving the main cost saving came from a small clinical audit. We recommended this finding was confirmed in a larger population. The method could be used to evaluate testing changes elsewhere. The results of the case study may be transferable to other jurisdictions where the organization of cancer services is fragmented

Economic evaluation of the introduction of the Prostate Health Index as a rule-out test to avoid unnecessary biopsies in men with prostate specific antigen levels of 4-10 in Hong Kong

A recent study showed that the Prostate Health Index may avoid unnecessary biopsies in men with prostate specific antigen 4-10ng/ml and normal digital rectal examination in the diagnosis of prostate cancer in Hong Kong. This study aimed to conduct an economic evaluation of the impact of adopting this commercially-available test in the Hong Kong public health service to determine whether further research is justified. A cost-consequence analysis was undertaken comparing the current diagnostic pathway with a proposed diagnostic pathway using the Prostate Health Index. Data for the model was taken from a prospective cohort study recruited at a single-institution and micro-costing studies. Using a cut off PHI score of 35 to avoid biopsy would cost HK$3,000 and save HK$7,988 per patient in biopsy costs and HK$511 from a reduction in biopsy-related adverse events. The net cost impact of the change was estimated to be HK$5,500 under base case assumptions. At the base case sensitivity and specificity for all grades of cancer (61.3% and 77.5% respectively) all grade cancer could be missed in 4.22% of the population and high grade cancer in 0.53%. The introduction of the prostate health index into the diagnostic pathway for prostate cancer in Hong Kong has the potential to reduce biopsies, biopsy costs and biopsy-related adverse events. Policy makers should consider the clinical and economic impact of this proposal

A case-controlled study of relatives’ complaints concerning patients who died in hospital: the role of treatment escalation / limitation planning

Objectives To independently assess quality of care among patients who died in hospital and whose next-of-kin submitted a letter of complaint and make comparisons with matched controls. To identify whether use of a treatment escalation limitation plan (TELP) during the terminal illness was a relevant background factor. Design The study was an investigator-blinded retrospective case-note review of 42 complaints cases and 72 controls matched for age, sex, ward location and time of death. Setting The acute medical and surgical wards of three District General Hospitals administered by NHS Lanarkshire, Scotland. Participants None. Intervention None. Outcome measures Quality of care: Clinical ‘problems’, non-beneficial interventions (NBIs) and harms were evaluated using the Structured Judgment Review Method. Complaints were categorized using the Healthcare Complaints Analysis Tool. Results The event frequencies and rate ratios for clinical ‘problems’, NBIs and harms were consistently higher in complaint cases compared to controls. The difference was only significant for NBIs (P = 0.05). TELPs were used less frequently in complaint cases compared to controls (23.8 versus 47.2%, P = 0.013). The relationship between TELP use and the three key clinical outcomes was nonsignificant. Conclusions Care delivered to patients at end-of-life whose next-of-kin submitted a complaint was poorer overall than among control patients when assessed independently by blinded reviewers. Regular use of a TELP in acute clinical settings has the potential to influence complaints relating to end-of-life care, but this requires further prospective study

Economic evaluation of genomic/genetic tests: a review and future directions

It has been suggested that health economists need to improve their methods in order to meet the challenges of evaluating genomic/genetic tests. In this article, we set out twelve challenges identified from a rapid review of the literature and suggest solutions to the challenges identified. Two challenges were common to all economic evaluations: choice of perspective and time-horizon. Five challenges were relevant for all diagnostic technologies: complexity of analysis; range of costs; under-developed evidence base; behavioral aspects; and choice of outcome metrics. The final five challenges were pertinent for genomic tests and only these may require methodological development: heterogeneity of tests and platforms, increasing stratification, capturing personal utility; incidental findings; and spillover effects. Current methods of economic evaluation are generally able to cope with genomic/genetic tests, although a renewed focus on specific decision-makers’ needs and a willingness to move away from cost-utility analysis may be required. Certain analysts may be constrained by reference cases developed primarily for the assessment of pharmaceuticals. The combined impact of multiple challenges may require analysts to be particularly careful in setting the scope of their analysis in order to ensure that feasibility is balanced with usefulness to the decision maker. A key issue is the under-developed evidence-base and it may be necessary to rethink translation processes to ensure sufficient, relevant evidence is available to support economic evaluation and adoption of genomic/genetic tests

Evidence Review: Assessment of COVID-19 in Primary Care: the Identification of Symptoms, Signs, Characteristics, Comorbidities and Clinical Signs in Adults which may Indicate a Higher Risk of Progression to Severe Disease

No abstract available