Use of Equivalent Relative Utility (ERU) to Evaluate Artificial Intelligence-Enabled Rule-Out Devices

We investigated the use of equivalent relative utility (ERU) to evaluate the effectiveness of artificial intelligence (AI)-enabled rule-out devices that use AI to identify and autonomously remove non-cancer patient images from radiologist review in screening mammography.We reviewed two performance metrics that can be used to compare the diagnostic performance between the radiologist-with-rule-out-device and radiologist-without-device workflows: positive/negative predictive values (PPV/NPV) and equivalent relative utility (ERU). To demonstrate the use of the two evaluation metrics, we applied both methods to a recent US-based study that reported an improved performance of the radiologist-with-device workflow compared to the one without the device by retrospectively applying their AI algorithm to a large mammography dataset. We further applied the ERU method to a European study utilizing their reported recall rates and cancer detection rates at different thresholds of their AI algorithm to compare the potential utility among different thresholds. For the study using US data, neither the PPV/NPV nor the ERU method can conclude a significant improvement in diagnostic performance for any of the algorithm thresholds reported. For the study using European data, ERU values at lower AI thresholds are found to be higher than that at a higher threshold because more false-negative cases would be ruled-out at higher threshold, reducing the overall diagnostic performance. Both PPV/NPV and ERU methods can be used to compare the diagnostic performance between the radiologist-with-device workflow and that without. One limitation of the ERU method is the need to measure the baseline, standard-of-care relative utility (RU) value for mammography screening in the US. Once the baseline value is known, the ERU method can be applied to large US datasets without knowing the true prevalence of the dataset

Natriuretic peptide activation of extracellular regulated kinase 1/2 (ERK1/2) pathway by particulate guanylyl cyclases in GH3 somatolactotropes.

The natriuretic peptides, Atrial-, B-type and C-type natriuretric peptides (ANP, BNP, CNP), are regulators of many endocrine tissues and exert their effects predominantly through the activation of their specific guanylyl cyclase receptors (GC-A and GC-B) to generate cGMP. Whereas cGMP-independent signalling has been reported in response to natriuretic peptides, this is mediated via either the clearance receptor (Npr-C) or a renal-specific NPR-Bi isoform, which both lack intrinsic guanylyl cyclase activity. Here, we report evidence of GC-B-dependent cGMP-independent signalling in pituitary GH3 cells. Stimulation of GH3 cells with CNP resulted in a rapid and sustained enhancement of ERK1/2 phosphorylation (P-ERK1/2), an effect that was not mimicked by dibutryl-cGMP. Furthermore, CNP-stimulated P-ERK1/2 occurred at concentrations below that required for cGMP accumulation. The effect of CNP on P-ERK1/2 was sensitive to pharmacological blockade of MEK (U0126) and Src kinases (PP2). Silencing of the GC-B1 and GC-B2 splice variants of the GC-B receptor by using targeted short interfering RNAs completely blocked the CNP effects on P-ERK1/2. CNP failed to alter GH3 cell proliferation or cell cycle distribution but caused a concentration-dependent increase in the activity of the human glycoprotein α-subunit promoter (αGSU) in a MEK-dependent manner. Finally, CNP also activated the p38 and JNK MAPK pathways in GH3 cells. These findings reveal an additional mechanism of GC-B signalling and suggest additional biological roles for CNP in its target tissues

Fiscal Year 2001

This report discusses the Low-Income Home Energy Assistance Program (LIHEAP), which is a block grant program under which the federal government provides states annual grants to operate multi-component home energy assistance programs for needy households

Inter-laboratory comparison of channelized hotelling observer computation

Item does not contain fulltex

Toward Fully Automated High-Resolution Electron Tomography

this paper, we describe our EMACT/ EMCAT system, which automates both tomographic data collection and reconstruction. 1996 Academic Press, In

Comparative Statistical Properties of Expected Utility and Area Under the ROC Curve for Laboratory Studies of Observer Performance in Screening Mammography

Rationale and objectivesOur objective is to determine whether expected utility (EU) and the area under the receiver operator characteristic (AUC) are consistent with one another as endpoints of observer performance studies in mammography. These two measures characterize receiver operator characteristic performance somewhat differently. We compare these two study endpoints at the level of individual reader effects, statistical inference, and components of variance across readers and cases.Materials and methodsWe reanalyze three previously published laboratory observer performance studies that investigate various x-ray breast imaging modalities using EU and AUC. The EU measure is based on recent estimates of relative utility for screening mammography.ResultsThe AUC and EU measures are correlated across readers for individual modalities (r = 0.93) and differences in modalities (r = 0.94 to 0.98). Statistical inference for modality effects based on multi-reader multi-case analysis is very similar, with significant results (P < .05) in exactly the same conditions. Power analyses show mixed results across studies, with a small increase in power on average for EU that corresponds to approximately a 7% reduction in the number of readers. Despite a large number of crossing receiver operator characteristic curves (59% of readers), modality effects only rarely have opposite signs for EU and AUC (6%).ConclusionsWe do not find any evidence of systematic differences between EU and AUC in screening mammography observer studies. Thus, when utility approaches are viable (i.e., an appropriate value of relative utility exists), practical effects such as statistical efficiency may be used to choose study endpoints

Inter-laboratory comparison of channelized hotelling observer computation

PURPOSE: The task-based assessment of image quality using model observers is increasingly used for the assessment of different imaging modalities. However, the performance computation of model observers needs standardization as well as a well-established trust in its implementation methodology and uncertainty estimation. The purpose of this work was to determine the degree of equivalence of the channelized Hotelling observer performance and uncertainty estimation using an intercomparison exercise. MATERIALS AND METHODS: Image samples to estimate model observer performance for detection tasks were generated from two-dimensional CT image slices of a uniform water phantom. A common set of images was sent to participating laboratories to perform and document the following tasks: (a) estimate the detectability index of a well-defined CHO and its uncertainty in three conditions involving different sized targets all at the same dose, and (b) apply this CHO to an image set where ground truth was unknown to participants (lower image dose). In addition, and on an optional basis, we asked the participating laboratories to (c) estimate the performance of real human observers from a psychophysical experiment of their choice. Each of the 13 participating laboratories was confidentially assigned a participant number and image sets could be downloaded through a secure server. Results were distributed with each participant recognizable by its number and then each laboratory was able to modify their results with justification as model observer calculation are not yet a routine and potentially error prone. RESULTS: Detectability index increased with signal size for all participants and was very consistent for 6 mm sized target while showing higher variability for 8 and 10 mm sized target. There was one order of magnitude between the lowest and the largest uncertainty estimation. CONCLUSIONS: This intercomparison helped define the state of the art of model observer performance computation and with thirteen participants, reflects openness and trust within the medical imaging community. The performance of a CHO with explicitly defined channels and a relatively large number of test images was consistently estimated by all participants. In contrast, the paper demonstrates that there is no agreement on estimating the variance of detectability in the training and testing setting.status: publishe