Setting clinical performance specifications to develop and evaluate biomarkers for clinical use

Background: Biomarker discovery studies often claim ‘promising’ findings, motivating further studies and marketing as medical tests. Unfortunately, the patient benefits promised are often inadequately explained to guide further evaluation, and few biomarkers have translated to improved patient care. We present a practical guide for setting minimum clinical performance specifications to strengthen clinical performance study design and interpretation. Methods: We developed a step-by-step approach using test evaluation and decision-analytic frameworks and present with illustrative examples. Results: We define clinical performance specifications as a set of criteria that quantify the clinical performance a new test must attain to allow better health outcomes than current practice. We classify the proposed patient benefits of a new test into three broad groups and describe how to set minimum clinical performance at the level where the potential harm of false-positive and false-negative results does not outweigh the benefits. (1) For add-on tests proposed to improve disease outcomes by improving detection, define an acceptable trade-off for false-positive versus true-positive results; (2) for triage tests proposed to reduce unnecessary tests and treatment by ruling out disease, define an acceptable risk of false-negatives as a safety threshold; (3) for replacement tests proposed to provide other benefits, or reduce costs, without compromising accuracy, use existing tests to benchmark minimum accuracy levels. Conclusions: Researchers can follow these guidelines to focus their study objectives and to define statistical hypotheses and sample size requirements. This way, clinical performance studies will allow conclusions about whether test performance is sufficient for intended use

Erratum to: Methods for evaluating medical tests and biomarkers

[This corrects the article DOI: 10.1186/s41512-016-0001-y.]

Evidence synthesis to inform model-based cost-effectiveness evaluations of diagnostic tests: a methodological systematic review of health technology assessments

Background: Evaluations of diagnostic tests are challenging because of the indirect nature of their impact on patient outcomes. Model-based health economic evaluations of tests allow different types of evidence from various sources to be incorporated and enable cost-effectiveness estimates to be made beyond the duration of available study data. To parameterize a health-economic model fully, all the ways a test impacts on patient health must be quantified, including but not limited to diagnostic test accuracy. Methods: We assessed all UK NIHR HTA reports published May 2009-July 2015. Reports were included if they evaluated a diagnostic test, included a model-based health economic evaluation and included a systematic review and meta-analysis of test accuracy. From each eligible report we extracted information on the following topics: 1) what evidence aside from test accuracy was searched for and synthesised, 2) which methods were used to synthesise test accuracy evidence and how did the results inform the economic model, 3) how/whether threshold effects were explored, 4) how the potential dependency between multiple tests in a pathway was accounted for, and 5) for evaluations of tests targeted at the primary care setting, how evidence from differing healthcare settings was incorporated. Results: The bivariate or HSROC model was implemented in 20/22 reports that met all inclusion criteria. Test accuracy data for health economic modelling was obtained from meta-analyses completely in four reports, partially in fourteen reports and not at all in four reports. Only 2/7 reports that used a quantitative test gave clear threshold recommendations. All 22 reports explored the effect of uncertainty in accuracy parameters but most of those that used multiple tests did not allow for dependence between test results. 7/22 tests were potentially suitable for primary care but the majority found limited evidence on test accuracy in primary care settings. Conclusions: The uptake of appropriate meta-analysis methods for synthesising evidence on diagnostic test accuracy in UK NIHR HTAs has improved in recent years. Future research should focus on other evidence requirements for cost-effectiveness assessment, threshold effects for quantitative tests and the impact of multiple diagnostic tests

Erratum to: Methods for evaluating medical tests and biomarkers

[This corrects the article DOI: 10.1186/s41512-016-0001-y.]

Effect of angiotensin-converting enzyme inhibitor and angiotensin receptor blocker initiation on organ support-free days in patients hospitalized with COVID-19

IMPORTANCE Overactivation of the renin-angiotensin system (RAS) may contribute to poor clinical outcomes in patients with COVID-19. Objective To determine whether angiotensin-converting enzyme (ACE) inhibitor or angiotensin receptor blocker (ARB) initiation improves outcomes in patients hospitalized for COVID-19. DESIGN, SETTING, AND PARTICIPANTS In an ongoing, adaptive platform randomized clinical trial, 721 critically ill and 58 non–critically ill hospitalized adults were randomized to receive an RAS inhibitor or control between March 16, 2021, and February 25, 2022, at 69 sites in 7 countries (final follow-up on June 1, 2022). INTERVENTIONS Patients were randomized to receive open-label initiation of an ACE inhibitor (n = 257), ARB (n = 248), ARB in combination with DMX-200 (a chemokine receptor-2 inhibitor; n = 10), or no RAS inhibitor (control; n = 264) for up to 10 days. MAIN OUTCOMES AND MEASURES The primary outcome was organ support–free days, a composite of hospital survival and days alive without cardiovascular or respiratory organ support through 21 days. The primary analysis was a bayesian cumulative logistic model. Odds ratios (ORs) greater than 1 represent improved outcomes. RESULTS On February 25, 2022, enrollment was discontinued due to safety concerns. Among 679 critically ill patients with available primary outcome data, the median age was 56 years and 239 participants (35.2%) were women. Median (IQR) organ support–free days among critically ill patients was 10 (–1 to 16) in the ACE inhibitor group (n = 231), 8 (–1 to 17) in the ARB group (n = 217), and 12 (0 to 17) in the control group (n = 231) (median adjusted odds ratios of 0.77 [95% bayesian credible interval, 0.58-1.06] for improvement for ACE inhibitor and 0.76 [95% credible interval, 0.56-1.05] for ARB compared with control). The posterior probabilities that ACE inhibitors and ARBs worsened organ support–free days compared with control were 94.9% and 95.4%, respectively. Hospital survival occurred in 166 of 231 critically ill participants (71.9%) in the ACE inhibitor group, 152 of 217 (70.0%) in the ARB group, and 182 of 231 (78.8%) in the control group (posterior probabilities that ACE inhibitor and ARB worsened hospital survival compared with control were 95.3% and 98.1%, respectively). CONCLUSIONS AND RELEVANCE In this trial, among critically ill adults with COVID-19, initiation of an ACE inhibitor or ARB did not improve, and likely worsened, clinical outcomes. TRIAL REGISTRATION ClinicalTrials.gov Identifier: NCT0273570

Mechanistic and structural studies of a novel l-proline dehydrogenase identified from the hyperthermophile pyrococcus furiosus DSM 3638

The cloning of two open reading frames encoding a heterodimeric protein related in sequence to bacterial sarcosine oxidase and dimethylglycine oxidase from hyperthermophilic Pyrococcus furiosus DSM 3638 is described. the protein was overexpressed in E. coli, purified to homogeneity and identified as a flavoprotein by virtue of the enzymes UV-visible absorption spectrum showing characteristic lambdamax at 367 and 450 nm. The physiological substrate was identified as L-proline (t ½ = ∼105 s-1 for bleaching of flavoprotein spectrum at 450 nm). Additionally, the enzyme oxidises L-pipecolic acid ( t½ = ∼110.5 s-1) and, to a lesser extent sarcosine (t½ = ∼654 s-1). No reactivity with sodium sulfite was detected, consistent with the enzyme belonging to the flavoprotein dehydrogenase class. These data classified the enzyme as a novel hyperthermophilic L-proline dehydrogenase.;The crystal structure of PRODH at 3.3 A resolution shows the enzyme is a heterooctamer (alphabeta)4. The holoenzyme contains one mol each of FAD, FMN and ATP per mol of alphabeta complex. Isolation of monoflavinylated enzyme containing a single FAD cofactor permitted detailed redox potentiometry and pH-dependence studies of the reaction with L-proline. A bell-shaped dependence for kcat/K m as a function of pH was observed with macroscopic pK a values (7.0 +/- 0.2 and 7.6 +/- 0.2) attributed to residue ionisations in the free enzyme. The pH dependence of k cat is sigmoidal, with maximum activity realised in the alkaline region; the dependence is described by a macroscopic pK a value of 7.7 +/- 0.1 and by analogy with other flavoenzymes is tentatively attributed to the ionisation of L-proline in the Michaelis complex. Studies with H225A, H225Q and Y251F mutants ruled out the participation of these residues as catalytic bases

Development of a new phosphorus partition relation for Australian steelmakers

Phosphorus is generally undesirable in steel. Decreasing availability of low phosphorus iron ores and the desire to recycle waste materials like basic oxygen steelmaking (BOS) slags is driving renewed interest in phosphorus removal. A number of phosphorus partition (LP) equations have been proposed in the literature for specific slag compositions and temperature ranges at equilibrium. These LP equations have been evaluated against the historic data on the phosphorus removal from an industrial top blown bottom stirred basic oxygen convertor. Further, the performance of these partition equations has been used to inform the development of a new LP model more suitable to the prevailing conditions in the Australian steelmaking industry. The new model has been used to isolate the key factors controlling dephosphorisation, namely lower temperature, higher basicity and higher oxygen potential. This LP model has allowed secondary factors influencing dephosphorisation to be assessed, including TiOx load, heat duration and stirring rate

A review of phosphorus partition relations for use in basic oxygen steelmaking

Phosphorus removal in basic oxygen steelmaking is a significant problem for integrated steelmakers. Phosphorus removal is required due to its deleterious effect on the mechanical properties of steel. However, this is progressively becoming more difficult due to the increasing phosphorus content of many iron ores. Many studies have investigated dephosphorisation and published empirical phosphorus partition (LP) equations for a range of conditions. The structure of these equations has been used to develop a new partition relation that allows the effect of minor slag constituents such as TiO2, Al2O3 and V2O5 on steel dephosphorisation to be tested. Al2O3 was found to have a weak negative effect on the measured LP, except at the lower oxygen potential range tested, where a positive correlation was observed. Increasing TiO2 and V2O5 contents were found to decrease the measured LP; however, these correlations became less prevalent at the higher oxygen potential ranges tested