A note on Youden's J and its cost ratio

<p>Abstract</p> <p>Background</p> <p>The Youden index, the sum of sensitivity and specificity minus one, is an index used for setting optimal thresholds on medical tests.</p> <p>Discussion</p> <p>When using this index, one implicitly uses decision theory with a ratio of misclassification costs which is equal to one minus the prevalence proportion of the disease. It is doubtful whether this cost ratio truly represents the decision maker's preferences. Moreover, in populations with a different prevalence, a selected threshold is optimal with reference to a different cost ratio.</p> <p>Summary</p> <p>The Youden index is not a truly optimal decision rule for setting thresholds because its cost ratio varies with prevalence. Researchers should look into their cost ratio and employ it in a decision theoretic framework to obtain genuinely optimal thresholds.</p

The predictive role of serum and bronchoalveolar lavage cytokines and adhesion molecules for acute respiratory distress syndrome development and outcome

BACKGROUND: The predictive role of many cytokines and adhesion molecules has not been studied systematically in acute respiratory distress syndrome (ARDS). METHODS: We measured prospectively tumour necrosis factor alpha (TNF-α), interleukin (IL)-1, soluble vascular adhesion molecule-1 (VCAM-1) and soluble intercellular adhesion molecule-1 (ICAM-1) in serum and bronchoalveolar lavage fluid (BALF) within 2 hours following admission, in 65 patients. The patients were divided into: those fulfilling the criteria for ARDS (n = 23, group A), those who were pre-ARDS and who developed ARDS within 24 hours (n = 14, group B), and those on pre-ARDS but who never developed ARDS (n = 28, group C). RESULTS: All the measured molecules were only found at higher levels in the serum of patients that died either with or without ARDS (P < 0.05 – P < 0.0001). Patients at risk exhibited a good negative predictive value (NPV) of the measured molecules for ARDS development both in their serum (89 to 95%) and BALF (86 to 92%) levels. In contrast to BALF, serum levels of IL-1 and adhesion molecules exhibited a good NPV (68 to 96%), sensitivity (60 to 88%) and survival specificity (74 to 96%) in all groups. All molecules in serum and BALF IL-1 were correlated with the APACHE II (P < 0.05 – P < 0.0001). Serum and BALF IL-1 as well as BALF TNF-α were negatively correlated to PaO(2)/FiO(2) (all P < 0.05). CONCLUSIONS: The studied molecules have good NPV for ARDS development both in serum and BALF. Serum rather than BALF levels seem to be related to outcome

The clinical significance of serum and bronchoalveolar lavage inflammatory cytokines in patients at risk for Acute Respiratory Distress Syndrome

BACKGROUND: The predictive role of many cytokines has not been well defined in Acute Respiratory Distress Syndrome (ARDS). METHODS: We measured prospectively IL-4, IL-6, IL-6 receptor, IL-8, and IL-10, in the serum and bronchoalveolar lavage fluid (BALF) in 59 patients who were admitted to ICU in order to identify predictive factors for the course and outcome of ARDS. The patients were divided into three groups: those fulfilling the criteria for ARDS (n = 20, group A), those at risk for ARDS and developed ARDS within 48 hours (n = 12, group B), and those at risk for ARDS but never developed ARDS (n = 27, group C). RESULTS: An excellent negative predictive value for ARDS development was found for IL-6 in BALF and serum (100% and 95%, respectively). IL-8 in BALF and IL-8 and IL-10 serum levels were higher in non-survivors in all studied groups, and were associated with a high negative predictive value. A significant correlation was found between IL-8 and APACHE score (r = 0.60, p < 0.0001). Similarly, IL-6 and IL-6r were highly correlated with PaO2/FiO2 (r = -0.27, p < 0.05 and r = -0.55, p < 0.0001, respectively). CONCLUSIONS: BALF and serum levels of the studied cytokines on admission may provide valuable information for ARDS development in patients at risk, and outcome in patients either in ARDS or in at risk for ARDS

A statistical framework to evaluate virtual screening

<p>Abstract</p> <p>Background</p> <p>Receiver operating characteristic (ROC) curve is widely used to evaluate virtual screening (VS) studies. However, the method fails to address the "early recognition" problem specific to VS. Although many other metrics, such as RIE, BEDROC, and pROC that emphasize "early recognition" have been proposed, there are no rigorous statistical guidelines for determining the thresholds and performing significance tests. Also no comparisons have been made between these metrics under a statistical framework to better understand their performances.</p> <p>Results</p> <p>We have proposed a statistical framework to evaluate VS studies by which the threshold to determine whether a ranking method is better than random ranking can be derived by bootstrap simulations and 2 ranking methods can be compared by permutation test. We found that different metrics emphasize "early recognition" differently. BEDROC and RIE are 2 statistically equivalent metrics. Our newly proposed metric SLR is superior to pROC. Through extensive simulations, we observed a "seesaw effect" – overemphasizing early recognition reduces the statistical power of a metric to detect true early recognitions.</p> <p>Conclusion</p> <p>The statistical framework developed and tested by us is applicable to any other metric as well, even if their exact distribution is unknown. Under this framework, a threshold can be easily selected according to a pre-specified type I error rate and statistical comparisons between 2 ranking methods becomes possible. The theoretical null distribution of SLR metric is available so that the threshold of SLR can be exactly determined without resorting to bootstrap simulations, which makes it easy to use in practical virtual screening studies.</p

Artificial Neural Networks Versus Multiple Logistic Regression to Predict 30-Day Mortality After Operations For Type A Ascending Aortic Dissection§

There are few comparative reports on the overall accuracy of neural networks (NN), assessed only versus multiple logistic regression (LR), to predict events in cardiovascular surgery studies and none has been performed among acute aortic dissection (AAD) Type A patients. OBJECTIVES: We aimed at investigating the predictive potential of 30-day mortality by a large series of risk factors in AAD Type A patients comparing the overall performance of NN versus LR. METHODS: We investigated 121 plus 87 AAD Type A patients consecutively operated during 7 years in two Centres. Forced and stepwise NN and LR solutions were obtained and compared, using receiver operating characteristic area under the curve (AUC) and their 95% confidence intervals (CI) and Gini's coefficients. Both NN and LR models were re-applied to data from the second Centre to adhere to a methodological imperative with NN. RESULTS: Forced LR solutions provided AUC 87.9+/-4.1% (CI: 80.7 to 93.2%) and 85.7+/-5.2% (CI: 78.5 to 91.1%) in the first and second Centre, respectively. Stepwise NN solution of the first Centre had AUC 90.5+/-3.7% (CI: 83.8 to 95.1%). The Gini's coefficients for LR and NN stepwise solutions of the first Centre were 0.712 and 0.816, respectively. When the LR and NN stepwise solutions were re-applied to the second Centre data, Gini's coefficients were, respectively, 0.761 and 0.850. Few predictors were selected in common by LR and NN models: the presence of pre-operative shock, intubation and neurological symptoms, immediate post-operative presence of dialysis in continuous and the quantity of post-operative bleeding in the first 24 h. The length of extracorporeal circulation, post-operative chronic renal failure and the year of surgery were specifically detected by NN. CONCLUSIONS: Different from the International Registry of AAD, operative and immediate post-operative factors were seen as potential predictors of short-term mortality. We report a higher overall predictive accuracy with NN than with LR. However, the list of potential risk factors to predict 30-day mortality after AAD Type A by NN model is not enlarged significantly

Metabolite Profiling of Alzheimer's Disease Cerebrospinal Fluid

Alzheimer's disease (AD) is a neurodegenerative disorder characterized by progressive loss of cognitive functions. Today the diagnosis of AD relies on clinical evaluations and is only late in the disease. Biomarkers for early detection of the underlying neuropathological changes are still lacking and the biochemical pathways leading to the disease are still not completely understood. The aim of this study was to identify the metabolic changes resulting from the disease phenotype by a thorough and systematic metabolite profiling approach. For this purpose CSF samples from 79 AD patients and 51 healthy controls were analyzed by gas and liquid chromatography-tandem mass spectrometry (GC-MS and LC-MS/MS) in conjunction with univariate and multivariate statistical analyses. In total 343 different analytes have been identified. Significant changes in the metabolite profile of AD patients compared to healthy controls have been identified. Increased cortisol levels seemed to be related to the progression of AD and have been detected in more severe forms of AD. Increased cysteine associated with decreased uridine was the best paired combination to identify light AD (MMSE>22) with specificity and sensitivity above 75%. In this group of patients, sensitivity and specificity above 80% were obtained for several combinations of three to five metabolites, including cortisol and various amino acids, in addition to cysteine and uridine

Serum Metabolomics Reveals Higher Levels of Polyunsaturated Fatty Acids in Lepromatous Leprosy: Potential Markers for Susceptibility and Pathogenesis

Leprosy is an infectious disease caused by the obligate intracellular bacterium Mycobacterium leprae. M. leprae infects the skin and nerves, leading to disfigurement and nerve damage, with the severity of the disease varying widely. We believe there are multiple factors (genetic, bacterial, nutritional and environmental), which may explain the differences in clinical manifestations of the disease. We studied the metabolites in the serum of infected patients to search for specific molecules that may contribute to variations in the severity of disease seen in leprosy. We found that there were variations in levels of certain lipids in the patients with different bacterial loads. In particular, we found that three polyunsaturated fatty acids (PUFAs) involved in the inhibition of inflammation were more abundant in the serum of patients with higher bacterial loads. However, we do not know whether these PUFAs originated from the host or the bacteria. The variations in the metabolite profile that we observed provide a foundation for future research into the explanations of how leprosy causes disease

Parts, Wholes, and Context in Reading: A Triple Dissociation

Research in object recognition has tried to distinguish holistic recognition from recognition by parts. One can also guess an object from its context. Words are objects, and how we recognize them is the core question of reading research. Do fast readers rely most on letter-by-letter decoding (i.e., recognition by parts), whole word shape, or sentence context? We manipulated the text to selectively knock out each source of information while sparing the others. Surprisingly, the effects of the knockouts on reading rate reveal a triple dissociation. Each reading process always contributes the same number of words per minute, regardless of whether the other processes are operating

Distinct Effector Memory CD4+ T Cell Signatures in Latent Mycobacterium tuberculosis Infection, BCG Vaccination and Clinically Resolved Tuberculosis

Two billion people worldwide are estimated to be latently infected with Mycobacterium tuberculosis (Mtb) and are at risk for developing active tuberculosis since Mtb can reactivate to cause TB disease in immune-compromised hosts. Individuals with latent Mtb infection (LTBI) and BCG-vaccinated individuals who are uninfected with Mtb, harbor antigen-specific memory CD4+ T cells. However, the differences between long-lived memory CD4+ T cells induced by latent Mtb infection (LTBI) versus BCG vaccination are unclear. In this study, we characterized the immune phenotype and functionality of antigen-specific memory CD4+ T cells in healthy BCG-vaccinated individuals who were either infected (LTBI) or uninfected (BCG) with Mtb. Individuals were classified into LTBI and BCG groups based on IFN-γ ELISPOT using cell wall antigens and ESAT-6/CFP-10 peptides. We show that LTBI individuals harbored high frequencies of late-stage differentiated (CD45RA−CD27−) antigen-specific effector memory CD4+ T cells that expressed PD-1. In contrast, BCG individuals had primarily early-stage (CD45RA−CD27+) cells with low PD-1 expression. CD27+ and CD27− as well as PD-1+ and PD-1− antigen-specific subsets were polyfunctional, suggesting that loss of CD27 expression and up-regulation of PD-1 did not compromise their capacity to produce IFN-γ, TNF-α and IL-2. PD-1 was preferentially expressed on CD27− antigen-specific CD4+ T cells, indicating that PD-1 is associated with the stage of differentiation. Using statistical models, we determined that CD27 and PD-1 predicted LTBI versus BCG status in healthy individuals and distinguished LTBI individuals from those who had clinically resolved Mtb infection after anti-tuberculosis treatment. This study shows that CD4+ memory responses induced by latent Mtb infection, BCG vaccination and clinically resolved Mtb infection are immunologically distinct. Our data suggest that differentiation into CD27−PD-1+ subsets in LTBI is driven by Mtb antigenic stimulation in vivo and that CD27 and PD-1 have the potential to improve our ability to evaluate true LTBI status