The behaviour of random forest permutation-based variable importance measures under predictor correlation

<p>Abstract</p> <p>Background</p> <p>Random forests (RF) have been increasingly used in applications such as genome-wide association and microarray studies where predictor correlation is frequently observed. Recent works on permutation-based variable importance measures (VIMs) used in RF have come to apparently contradictory conclusions. We present an extended simulation study to synthesize results.</p> <p>Results</p> <p>In the case when both predictor correlation was present and predictors were associated with the outcome (H_A), the unconditional RF VIM attributed a higher share of importance to correlated predictors, while under the null hypothesis that no predictors are associated with the outcome (H₀) the unconditional RF VIM was unbiased. Conditional VIMs showed a decrease in VIM values for correlated predictors versus the unconditional VIMs under H_Aand was unbiased under H₀. Scaled VIMs were clearly biased under H_Aand H₀.</p> <p>Conclusions</p> <p>Unconditional unscaled VIMs are a computationally tractable choice for large datasets and are unbiased under the null hypothesis. Whether the observed increased VIMs for correlated predictors may be considered a "bias" - because they do not directly reflect the coefficients in the generating model - or if it is a beneficial attribute of these VIMs is dependent on the application. For example, in genetic association studies, where correlation between markers may help to localize the functionally relevant variant, the increased importance of correlated predictors may be an advantage. On the other hand, we show examples where this increased importance may result in spurious signals.</p

Human Double-Negative Regulatory T-Cells Induce a Metabolic and Functional Switch in Effector T-Cells by Suppressing mTOR Activity

The recently discovered population of TCRαβ+ CD4–/CD8– (double-negative, DN) T-cells are highly potent suppressor cells in mice and humans. In preclinical transplantation models, adoptive transfer of DN T-cells specifically inhibits alloreactive T-cells and prevents transplant rejection or graft-vs.-host disease (GvHD). Interestingly, clinical studies in patients who underwent allogeneic stem cell transplantation reveal an inverse correlation between the frequency of circulating DN T-cells and the severity of GvHD, suggesting a therapeutic potential of human DN T-cells. However, their exact mode of action has not been elucidated yet. Investigating the impact of DN T-cells on conventional T-cells, we found that human DN T-cells selectively inhibit mTOR signaling in CD4 T-cells. Given that mTOR is a critical regulator of cellular metabolism, we further determined the impact of DN T-cells on the metabolic framework of T-cells. Intriguingly, DN T-cells diminished expression of glucose transporters and glucose uptake, whereas fatty acid uptake was not modified, indicating that DN T-cells prevent metabolic adaptation of CD4 T-cells upon activation (i.e., glycolytic switch) thereby contributing to their suppression. Further analyses demonstrated that CD4 T-cells also do not upregulate homing receptors associated with inflammatory processes. In contrast, expression of central memory-cell associated cell surface markers and transcription factors were increased by DN T-cells. Moreover, CD4 T-cells failed to produce inflammatory cytokines after co-culture with DN T-cells, whereas IL-2 secretion was enhanced. Taken together DN T-cells impair metabolic reprogramming of conventional CD4 T-cells by abrogating mTOR signaling, thereby modulating CD4 T-cell functionality. These results uncover a new mechanism of DN T-cell-mediated suppression, pointing out that DN T-cells could serve as cell-based therapy to limit alloreactive immune response

Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis

In this crowdsourced initiative, independent analysts used the same dataset to test two hypotheses regarding the effects of scientists’ gender and professional status on verbosity during group meetings. Not only the analytic approach but also the operationalizations of key variables were left unconstrained and up to individual analysts. For instance, analysts could choose to operationalize status as job title, institutional ranking, citation counts, or some combination. To maximize transparency regarding the process by which analytic choices are made, the analysts used a platform we developed called DataExplained to justify both preferred and rejected analytic paths in real time. Analyses lacking sufficient detail, reproducible code, or with statistical errors were excluded, resulting in 29 analyses in the final sample. Researchers reported radically different analyses and dispersed empirical outcomes, in a number of cases obtaining significant effects in opposite directions for the same research question. A Boba multiverse analysis demonstrates that decisions about how to operationalize variables explain variability in outcomes above and beyond statistical choices (e.g., covariates). Subjective researcher decisions play a critical role in driving the reported empirical results, underscoring the need for open data, systematic robustness checks, and transparency regarding both analytic paths taken and not taken. Implications for organizations and leaders, whose decision making relies in part on scientific findings, consulting reports, and internal analyses by data scientists, are discussed