Effect Size Estimation and Misclassification Rate Based Variable Selection in Linear Discriminant Analysis

Supervised classifying of biological samples based on genetic information, (e.g. gene expression profiles) is an important problem in biostatistics. In order to find both accurate and interpretable classification rules variable selection is indispensable. This article explores how an assessment of the individual importance of variables (effect size estimation) can be used to perform variable selection. I review recent effect size estimation approaches in the context of linear discriminant analysis (LDA) and propose a new conceptually simple effect size estimation method which is at the same time computationally efficient. I then show how to use effect sizes to perform variable selection based on the misclassification rate which is the data independent expectation of the prediction error. Simulation studies and real data analyses illustrate that the proposed effect size estimation and variable selection methods are competitive. Particularly, they lead to both compact and interpretable feature sets.Comment: 21 pages, 2 figure

Predictive physiological anticipatory activity preceding seemingly unpredictable stimuli: An update of Mossbridge et al\u2019s meta-analysis

Background: This is an update of the Mossbridge et al\u2019s meta-analysis related to the physiological anticipation preceding seemingly unpredictable stimuli which overall effect size was 0.21; 95% Confidence Intervals: 0.13 - 0.29 Methods: Nineteen new peer and non-peer reviewed studies completed from January 2008 to June 2018 were retrieved describing a total of 27 experiments and 36 associated effect sizes. Results: The overall weighted effect size, estimated with a frequentist multilevel random model, was: 0.28; 95% Confidence Intervals: 0.18-0.38; the overall weighted effect size, estimated with a multilevel Bayesian model, was: 0.28; 95% Credible Intervals: 0.18-0.38. The weighted mean estimate of the effect size of peer reviewed studies was higher than that of non-peer reviewed studies, but with overlapped confidence intervals: Peer reviewed: 0.36; 95% Confidence Intervals: 0.26-0.47; Non-Peer reviewed: 0.22; 95% Confidence Intervals: 0.05-0.39. Similarly, the weighted mean estimate of the effect size of Preregistered studies was higher than that of Non-Preregistered studies: Preregistered: 0.31; 95% Confidence Intervals: 0.18-0.45; No-Preregistered: 0.24; 95% Confidence Intervals: 0.08-0.41. The statistical estimation of the publication bias by using the Copas selection model suggest that the main findings are not contaminated by publication bias. Conclusions: In summary, with this update, the main findings reported in Mossbridge et al\u2019s meta-analysis, are confirmed

Multivariate methods and small sample size: combining with small effect size

This manuscript is the author's response to: "Dochtermann, N.A. & Jenkins, S.H. Multivariate methods and small sample\ud sizes, Ethology, 117, 95-101." and accompanies this paper: "Budaev, S. Using principal components and factor analysis in animal behaviour research: Caveats and guidelines. Ethology, 116, 472-480"\u

Anchorage Community Survey 2007 Survey Sampling Design: Power and Sample Size

This working paper documents the power analysis, literature review, and precision considerations contemplated in designing the Anchorage Community Survey’s (ACS) 2007 sampling design. The ACS will obtain at least 30 completed surveys from individuals in each of the 55 census tracts that make up the Anchorage Municipality, allowing us to discern a fairly small effect size of 0.30 with our smallest anticipated intraclass correlation and a moderate effect size of 0.40 with our largest anticipated intraclass correlation, both at 0.80 power level. This cluster sample size and number of clusters should yield sufficient precision to allow good estimation of variance components and standard errors, acceptable reliability estimates, and reasonable aggregated measures of constructed neighborhood variables from individual survey item responses.Abstract / Introduction / Number of clusters (J) = 55 / Cluster Size (n) = 30 / Intraclass correlation (ρ)=.10 to .20 / Effect size (δ)=.30 or greater / Power Graphs / Support from the Literature / A Note on Precision / Reference

Multiple components of environmental change drive populations of breeding waders in seminatural grasslands

Environments are rapidly changing due to climate change, land use, intensive agriculture, and the impact of hunting on predator populations. Here, we analyzed longterm data recorded during 1928–2014 on the size of breeding populations of waders at two large nature reserves in Denmark, Vejlerne and Tipperne, to determine the effects of components of environmental change on breeding populations of waders. Environmental variables and counts of waders were temporally autocorrelated, and we used generalized least square (GLS) by incorporating the first-order autoregressive correlation structure in the analyses. We attempted to predict the abundance of waders for short-term trends for two nature reserves (35 years) and for long-term trends for one nature reserve (86 years), using precipitation, temperature, nutrients, abundance of foxes Vulpes vulpes, area grazed, and number of cattle. There was evidence of impacts of nutrients, climate (long-term changes in temperature and precipitation), grazing, mowing, and predation on bird populations. We used standard random effects meta-analyses weighted by (N–3) to quantify these mean effects. There was no significant difference in effect size among species, while mean effect size differed consistently among environmental factors, and the interaction between effect size for species and environmental factors was also significant. Thus, environmental factors affected the different species differently. Mean effect size was the largest at +0.20 for rain, +0.11 for temperature, −0.09 for fox abundance, and −0.03 for number of cattle, while there was no significant mean effect for fertilizer, area grazed, and year. Effect sizes for two short-term time series from Tipperne and Vejlerne were positively correlated as were effect sizes for short-term and long-term time series at Tipperne. This implies that environmental factors had consistent effects across large temporal and spatial scales

Standardized or simple effect size: what should be reported?

It is regarded as best practice for psychologists to report effect size when disseminating quantitative research findings. Reporting of effect size in the psychological literature is patchy – though this may be changing – and when reported it is far from clear that appropriate effect size statistics are employed. This paper considers the practice of reporting point estimates of standardized effect size and explores factors such as reliability, range restriction and differences in design that distort standardized effect size unless suitable corrections are employed. For most purposes simple (unstandardized) effect size is more robust and versatile than standardized effect size. Guidelines for deciding what effect size metric to use and how to report it are outlined. Foremost among these are: i) a preference for simple effect size over standardized effect size, and ii) the use of confidence intervals to indicate a plausible range of values the effect might take. Deciding on the appropriate effect size statistic to report always requires careful thought and should be influenced by the goals of the researcher, the context of the research and the potential needs of readers

Feeling the future: A meta-analysis of 90 experiments on the anomalous anticipation of random future events

In 2011, one of the authors (DJB) published a report of nine experiments in the Journal of Personality and Social Psychology purporting to demonstrate that an individual\u2019s cognitive and affective responses can be influenced by randomly selected stimulus events that do not occur until after his or her responses have already been made and recorded, a generalized variant of the phenomenon traditionally denoted by the term precognition. To encourage replications, all materials needed to conduct them were made available on request. We here report a meta-analysis of 90 experiments from 33 laboratories in 14 countries which yielded an overall effect greater than 6 sigma, z = 6.40, p = 1.2 7 10 with an effect size (Hedges\u2019 g) of 0.09. A Bayesian analysis yielded a Bayes Factor of 5.1 7 10 , greatly exceeding the criterion value of 100 for \u201cdecisive evidence\u201d in support of the experimental hypothesis. When DJB\u2019s original experiments are excluded, the combined effect size for replications by independent investigators is 0.06, z = 4.16, p = 1.1 7 10 , and the BF value is 3,853, again exceeding the criterion for \u201cdecisive evidence.\u201d The number of potentially unretrieved experiments required to reduce the overall effect size of the complete database to a trivial value of 0.01 is 544, and seven of eight additional statistical tests support the conclusion that the database is not significantly compromised by either selection bias or by intense \u201cp -hacking\u201d\u2014the selective suppression of findings or analyses that failed to yield statistical significance. P-curve analysis, a recently introduced statistical technique, estimates the true effect size of the experiments to be 0.20 for the complete database and 0.24 for the independent replications, virtually identical to the effect size of DJB\u2019s original experiments (0.22) and the closely related \u201cpresentiment\u201d experiments (0.21). We discuss the controversial status of precognition and other anomalous effects collectively known as psi