Net Reclassification Index: a Misleading Measure of Prediction Improvement

The evaluation of biomarkers to improve risk prediction is a common theme in modern research. Since its introduction in 2008, the net reclassification index (NRI) (Pencina et al. 2008, Pencina et al. 2011) has gained widespread use as a measure of prediction performance with over 1,200 citations as of June 30, 2013. The NRI is considered by some to be more sensitive to clinically important changes in risk than the traditional change in the AUC (Delta AUC) statistic (Hlatky et al. 2009). Recent statistical research has raised questions, however, about the validity of conclusions based on the NRI. (Hilden and Gerds 2013, Pepe et al. 2013) Here we illustrate one serious problem, that unlike classic measures of prediction performance, the NRI can provide a biased assessment of prediction performance even with independent validation data

Validating Five Questions of Antiretroviral Nonadherence in a Public-Sector Treatment Program in Rural South Africa

Simple questions are the most commonly used measures of antiretroviral treatment (ART) adherence in sub-Saharan Africa (SSA), but rarely validated. We administered five adherence questions in a public-sector primary care clinic in rural South Africa: 7-day recall of missed doses, 7-day recall of late doses, a six-level Likert item, a 30-day visual analogue scale of the proportion of doses missed, and recall of the time when an ART dose was last missed. We estimated question sensitivity and specificity in detecting immunologic (or virologic) failure assessed within 45 days of the adherence question date. Of 165 individuals, 7% had immunologic failure; 137 individuals had viral loads with 9% failure detected. The Likert item performed best for immunologic failure with sensitivity/specificity of 100%/5% (when defining nonadherence as self-reported adherence less than -excellent-), 42%/55% (less than -very good-), and 25%/95% (less than -good-). The remaining questions had sensitivities <=17%, even when the least strict cutoffs defined nonadherence. When we stratified the analysis by gender, age, or education, question performance was not substantially better in any of the subsamples in comparison to the total sample. Five commonly used adherence questions performed poorly in identifying patients with treatment failure in a public-sector ART program in SSA. Valid adherence measurement instruments are urgently required to identify patients needing treatment support and those most at risk of treatment failure. Available estimates of ART adherence in SSA are mostly based on studies using adherence questions. It is thus unlikely that our understanding of ART adherence in the region is correct.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/90502/1/apc-2E2010-2E0257.pd

Should We Abandon the t-Test in the Analysis of Gene Expression Microarray Data: A Comparison of Variance Modeling Strategies

High-throughput post-genomic studies are now routinely and promisingly investigated in biological and biomedical research. The main statistical approach to select genes differentially expressed between two groups is to apply a t-test, which is subject of criticism in the literature. Numerous alternatives have been developed based on different and innovative variance modeling strategies. However, a critical issue is that selecting a different test usually leads to a different gene list. In this context and given the current tendency to apply the t-test, identifying the most efficient approach in practice remains crucial. To provide elements to answer, we conduct a comparison of eight tests representative of variance modeling strategies in gene expression data: Welch's t-test, ANOVA [1], Wilcoxon's test, SAM [2], RVM [3], limma [4], VarMixt [5] and SMVar [6]. Our comparison process relies on four steps (gene list analysis, simulations, spike-in data and re-sampling) to formulate comprehensive and robust conclusions about test performance, in terms of statistical power, false-positive rate, execution time and ease of use. Our results raise concerns about the ability of some methods to control the expected number of false positives at a desirable level. Besides, two tests (limma and VarMixt) show significant improvement compared to the t-test, in particular to deal with small sample sizes. In addition limma presents several practical advantages, so we advocate its application to analyze gene expression data

Derivation and validation of a risk-factor model for detection of oral potentially malignant disorders in populations with high prevalence

Background:Oral and pharyngeal cancers constitute the sixth most common type of cancer globally, with high morbidity and mortality. In many countries, most cases of oral cancer arise from long-standing, pre-existing lesions, yet advanced malignancies prevail. A new approach to early detection is needed. We aimed to validate a model for screening so that only high-risk individuals receive the clinical examination.Methods:A community-based case-control study (n1029) in rural Sri Lanka assessed risk factors and markers for oral potentially malignant disorders (OPMD) by administering a questionnaire followed by an oral examination. We then developed a model based on age, socioeconomic status and habits of betel-quid chewing, alcohol drinking and tobacco smoking, with weightings based on odds ratios from the multiple logistic regression. A total, single score was calculated per individual. Standard receiver-operator characteristic curves were plotted for the total score and presence of OPMD. The model was validated on a new sample of 410 subjects in a different community.Results:A score of 12.0 produced optimal sensitivity (95.5%), specificity (75.9%), false-positive rate (24.0%), false-negative rate (4.5%), positive predictive value (35.9%) and negative predictive value (99.2%).Conclusion:This model is suitable for detection of OPMD and oral cancer in high-risk communities, for example, in Asia, the Pacific and the global diaspora therefrom. A combined risk-factor score of 12.0 was optimal for participation in oral cancer/OPMD screening in Sri Lanka. The model, or local adaptations, should have wide applicability

The IceCube Neutrino Observatory, the Pierre Auger Observatory and the Telescope Array: Joint Contribution to the 34th International Cosmic Ray Conference (ICRC 2015)

We have conducted three searches for correlations between ultra-high energy cosmic rays detected by the Telescope Array and the Pierre Auger Observatory, and high-energy neutrino candidate events from IceCube. Two cross-correlation analyses with UHECRs are done: one with 39 cascades from the IceCube `high-energy starting events' sample and the other with 16 high-energy `track events'. The angular separation between the arrival directions of neutrinos and UHECRs is scanned over. The same events are also used in a separate search using a maximum likelihood approach, after the neutrino arrival directions are stacked. To estimate the significance we assume UHECR magnetic deflections to be inversely proportional to their energy, with values $3^\circ$, $6^\circ$ and $9^\circ$ at 100 EeV to allow for the uncertainties on the magnetic field strength and UHECR charge. A similar analysis is performed on stacked UHECR arrival directions and the IceCube sample of through-going muon track events which were optimized for neutrino point-source searches.Comment: one proceeding, the 34th International Cosmic Ray Conference, 30 July - 6 August 2015, The Hague, The Netherlands; will appear in PoS(ICRC2015

The IceCube Neutrino Observatory, the Pierre Auger Observatory and the Telescope Array:Joint Contribution to the 34th International Cosmic Ray Conference (ICRC 2015)

We have conducted three searches for correlations between ultra-high energy cosmic rays detected by the Telescope Array and the Pierre Auger Observatory, and high-energy neutrino candidate events from IceCube. Two cross-correlation analyses with UHECRs are done: one with 39 cascades from the IceCube `high-energy starting events' sample and the other with 16 high-energy `track events'. The angular separation between the arrival directions of neutrinos and UHECRs is scanned over. The same events are also used in a separate search using a maximum likelihood approach, after the neutrino arrival directions are stacked. To estimate the significance we assume UHECR magnetic deflections to be inversely proportional to their energy, with values $3^\circ$, $6^\circ$ and $9^\circ$ at 100 EeV to allow for the uncertainties on the magnetic field strength and UHECR charge. A similar analysis is performed on stacked UHECR arrival directions and the IceCube sample of through-going muon track events which were optimized for neutrino point-source searches

Consistent patterns of common species across tropical tree communities

Trees structure the Earth’s most biodiverse ecosystem, tropical forests. The vast number of tree species presents a formidable challenge to understanding these forests, including their response to environmental change, as very little is known about most tropical tree species. A focus on the common species may circumvent this challenge. Here we investigate abundance patterns of common tree species using inventory data on 1,003,805 trees with trunk diameters of at least 10 cm across 1,568 locations1,2,3,4,5,6 in closed-canopy, structurally intact old-growth tropical forests in Africa, Amazonia and Southeast Asia. We estimate that 2.2%, 2.2% and 2.3% of species comprise 50% of the tropical trees in these regions, respectively. Extrapolating across all closed-canopy tropical forests, we estimate that just 1,053 species comprise half of Earth’s 800 billion tropical trees with trunk diameters of at least 10 cm. Despite differing biogeographic, climatic and anthropogenic histories7, we find notably consistent patterns of common species and species abundance distributions across the continents. This suggests that fundamental mechanisms of tree community assembly may apply to all tropical forests. Resampling analyses show that the most common species are likely to belong to a manageable list of known species, enabling targeted efforts to understand their ecology. Although they do not detract from the importance of rare species, our results open new opportunities to understand the world’s most diverse forests, including modelling their response to environmental change, by focusing on the common species that constitute the majority of their trees.Publisher PDFPeer reviewe