The Significance Filter, the Winner's Curse and the Need to Shrink

The "significance filter" refers to focusing exclusively on statistically significant results. Since frequentist properties such as unbiasedness and coverage are valid only before the data have been observed, there are no guarantees if we condition on significance. In fact, the significance filter leads to overestimation of the magnitude of the parameter, which has been called the "winner's curse". It can also lead to undercoverage of the confidence interval. Moreover, these problems become more severe if the power is low. While these issues clearly deserve our attention, they have been studied only informally and mathematical results are lacking. Here we study them from the frequentist and the Bayesian perspective. We prove that the relative bias of the magnitude is a decreasing function of the power and that the usual confidence interval undercovers when the power is less than 50%. We conclude that failure to apply the appropriate amount of shrinkage can lead to misleading inferences

Psychological and behavioural impact of returning personal results from whole-genome sequencing: the HealthSeq project

Providing ostensibly healthy individuals with personal results from whole-genome sequencing could lead to improved health and well-being via enhanced disease risk prediction, prevention, and diagnosis, but also poses practical and ethical challenges. Understanding how individuals react psychologically and behaviourally will be key in assessing the potential utility of personal whole-genome sequencing. We conducted an exploratory longitudinal cohort study in which quantitative surveys and in-depth qualitative interviews were conducted before and after personal results were returned to individuals who underwent whole-genome sequencing. The participants were offered a range of interpreted results, including Alzheimer’s disease, type 2 diabetes, pharmacogenomics, rare disease-associated variants, and ancestry. They were also offered their raw data. Of the 35 participants at baseline, 29 (82.9%) completed the 6-month follow-up. In the quantitative surveys, test-related distress was low, although it was higher at 1-week than 6-month follow-up (Z=2.68, P=0.007). In the 6-month qualitative interviews, most participants felt happy or relieved about their results. A few were concerned, particularly about rare disease-associated variants and Alzheimer’s disease results. Two of the 29 participants had sought clinical follow-up as a direct or indirect consequence of rare disease-associated variants results. Several had mentioned their results to their doctors. Some participants felt having their raw data might be medically useful to them in the future. The majority reported positive reactions to having their genomes sequenced, but there were notable exceptions to this. The impact and value of returning personal results from whole-genome sequencing when implemented on a larger scale remains to be seen

Assessing Professionalism: A theoretical framework for defining clinical rotation assessment criteria

Although widely accepted as an important graduate competence, professionalism is a challenging outcome to define and assess. Clinical rotations provide an excellent opportunity to develop student professionalism through the use of experiential learning and effective feedback, but without appropriate theoretical frameworks, clinical teachers may find it difficult to identify appropriate learning outcomes. The adage “I know it when I see it” is unhelpful in providing feedback and guidance for student improvement, and criteria that are more specifically defined would help students direct their own development. This study sought first to identify how clinical faculty in one institution currently assess professionalism, using retrospective analysis of material obtained in undergraduate teaching and faculty development sessions. Subsequently, a faculty workshop was held in which a round-table type discussion sought to develop these ideas and identify how professionalism assessment could be improved. The output of this session was a theoretical framework for teaching and assessing professionalism, providing example assessment criteria and ideas for clinical teaching. This includes categories such as client and colleague interaction, respect and trust, recognition of limitations, and understanding of different professional identities. Each category includes detailed descriptions of the knowledge, skills, and behaviors expected of students in these areas. The criteria were determined by engaging faculty in the development of the framework, and therefore they should represent a focused development of criteria already used to assess professionalism, and not a novel and unfamiliar set of assessment guidelines. The faculty-led nature of this framework is expected to facilitate implementation in clinical teaching

Linear regression for numeric symbolic variables: an ordinary least squares approach based on Wasserstein Distance

In this paper we present a linear regression model for modal symbolic data. The observed variables are histogram variables according to the definition given in the framework of Symbolic Data Analysis and the parameters of the model are estimated using the classic Least Squares method. An appropriate metric is introduced in order to measure the error between the observed and the predicted distributions. In particular, the Wasserstein distance is proposed. Some properties of such metric are exploited to predict the response variable as direct linear combination of other independent histogram variables. Measures of goodness of fit are discussed. An application on real data corroborates the proposed method

Null hypothesis significance testing: a short tutorial

Although thoroughly criticized, null hypothesis significance testing (NHST) remains the statistical method of choice used to provide evidence for an effect, in biological, biomedical and social sciences. In this short tutorial, I first summarize the concepts behind the method, distinguishing test of significance (Fisher) and test of acceptance (Newman-Pearson) and point to common interpretation errors regarding the p-value. I then present the related concepts of confidence intervals and again point to common interpretation errors. Finally, I discuss what should be reported in which context. The goal is to clarify concepts to avoid interpretation errors and propose reporting practices.</ns4:p

Mentoring at the University of Pennsylvania: Results of a Faculty Survey

BACKGROUND: Research suggests mentoring is related to career satisfaction and success. Most studies have focused on junior faculty. OBJECTIVE: To explore multiple aspects of mentoring at an academic medical center in relation to faculty rank, track, and gender. DESIGN: Cross-sectional mail survey in mid-2003. PARTICIPANTS: Faculty members, 1,432, at the University of Pennsylvania School of Medicine MEASUREMENTS: Self-administered survey developed from existing instruments and stakeholders. RESULTS: Response rate was 73% (n = 1,046). Most (92%) assistant and half (48%) of associate professors had a mentor. Assistant professors in the tenure track were most likely to have a mentor (98%). At both ranks, the faculty was given more types of advice than types of opportunities. Satisfaction with mentoring was correlated with the number of types of mentoring received (r = .48 and .53, P < .0001), job satisfaction (r = .44 and .31, P < .0001), meeting frequency (r = .53 and .61, P < .0001), and expectation of leaving the University within 5 years (Spearman r = −.19 and −.18, P < .0001), at the assistant and associate rank, respectively. Significant predictors of higher overall job satisfaction were associate rank [Odds ratio (OR) = 2.04, CI = 1.29–3.21], the 10-point mentoring satisfaction rating (OR = 1.27, CI = 1.17–1.35), and number of mentors (OR = 1.60, CI = 1.20–2.07). CONCLUSIONS: Having a mentor, or preferably, multiple mentors is strongly related to satisfaction with mentoring and overall job satisfaction. Surprisingly, few differences were related to gender. Mentoring of clinician–educators, research track faculty, and senior faculty, and the use of multiple mentors require specific attention of academic leadership and further study

Analytic philosophy for biomedical research: the imperative of applying yesterday's timeless messages to today's impasses

The mantra that "the best way to predict the future is to invent it" (attributed to the computer scientist Alan Kay) exemplifies some of the expectations from the technical and innovative sides of biomedical research at present. However, for technical advancements to make real impacts both on patient health and genuine scientific understanding, quite a number of lingering challenges facing the entire spectrum from protein biology all the way to randomized controlled trials should start to be overcome. The proposal in this chapter is that philosophy is essential in this process. By reviewing select examples from the history of science and philosophy, disciplines which were indistinguishable until the mid-nineteenth century, I argue that progress toward the many impasses in biomedicine can be achieved by emphasizing theoretical work (in the true sense of the word 'theory') as a vital foundation for experimental biology. Furthermore, a philosophical biology program that could provide a framework for theoretical investigations is outlined