Evolution of statistical analysis in empirical software engineering research: Current state and steps forward

Software engineering research is evolving and papers are increasingly based on empirical data from a multitude of sources, using statistical tests to determine if and to what degree empirical evidence supports their hypotheses. To investigate the practices and trends of statistical analysis in empirical software engineering (ESE), this paper presents a review of a large pool of papers from top-ranked software engineering journals. First, we manually reviewed 161 papers and in the second phase of our method, we conducted a more extensive semi-automatic classification of papers spanning the years 2001--2015 and 5,196 papers. Results from both review steps was used to: i) identify and analyze the predominant practices in ESE (e.g., using t-test or ANOVA), as well as relevant trends in usage of specific statistical methods (e.g., nonparametric tests and effect size measures) and, ii) develop a conceptual model for a statistical analysis workflow with suggestions on how to apply different statistical methods as well as guidelines to avoid pitfalls. Lastly, we confirm existing claims that current ESE practices lack a standard to report practical significance of results. We illustrate how practical significance can be discussed in terms of both the statistical analysis and in the practitioner's context.Comment: journal submission, 34 pages, 8 figure

Balancing generalization and lexical conservatism : an artificial language study with child learners

Successful language acquisition involves generalization, but learners must balance this against the acquisition of lexical constraints. Such learning has been considered problematic for theories of acquisition: if learners generalize abstract patterns to new words, how do they learn lexically-based exceptions? One approach claims that learners use distributional statistics to make inferences about when generalization is appropriate, a hypothesis which has recently received support from Artificial Language Learning experiments with adult learners (Wonnacott, Newport, & Tanenhaus, 2008). Since adult and child language learning may be different (Hudson Kam & Newport, 2005), it is essential to extend these results to child learners. In the current work, four groups of children (6 years) were each exposed to one of four semi-artificial languages. The results demonstrate that children are sensitive to linguistic distributions at and above the level of particular lexical items, and that these statistics influence the balance between generalization and lexical conservatism. The data are in line with an approach which models generalization as rational inference and in particular with the predictions of the domain general hierarchical Bayesian model developed in Kemp, Perfors & Tenenbaum, 2006. This suggests that such models have relevance for theories of language acquisition

Modeling Binary Time Series Using Gaussian Processes with Application to Predicting Sleep States

Motivated by the problem of predicting sleep states, we develop a mixed effects model for binary time series with a stochastic component represented by a Gaussian process. The fixed component captures the effects of covariates on the binary-valued response. The Gaussian process captures the residual variations in the binary response that are not explained by covariates and past realizations. We develop a frequentist modeling framework that provides efficient inference and more accurate predictions. Results demonstrate the advantages of improved prediction rates over existing approaches such as logistic regression, generalized additive mixed model, models for ordinal data, gradient boosting, decision tree and random forest. Using our proposed model, we show that previous sleep state and heart rates are significant predictors for future sleep states. Simulation studies also show that our proposed method is promising and robust. To handle computational complexity, we utilize Laplace approximation, golden section search and successive parabolic interpolation. With this paper, we also submit an R-package (HIBITS) that implements the proposed procedure.Comment: Journal of Classification (2018

Perception threshold and electrode position for spinal cord stimulation

The perception threshold for epidural spinal cord stimulation in chronic pain management was analyzed on 3923 testing data obtained from 136 implanted patients. The initial areas of paresthesiae due to stimulation were recorded and reported as the stimulation map according to the location of electrodes. Measurement of dorsal thickness of the cerebrospinal fluid (CSF) layer was obtained from 26 subjects using magnetic resonance imaging (MRI). The results indicate that the perception threshold is a function of the spinal level of the implanted electrodes, of the mediolateral position in the spinal canal and the contact separation of electrode. Differences in perception threshold at various vertebral levels are mainly due to varying depths of the dorsal CSF layer. The medially placed electrodes caudal to the mid-cervical levels have a higher perception threshold than more laterally placed ones. The electrodes at high and mid-cervical levels, however, have a smaller perception threshold if placed medially. The information obtained from this investigation has important implications for the design of a new-generation stimulation system and clinical application to maximize the longevity of the power source

Analysis of interference to cable television due to mobile usage in the Digital Dividend

The start of use of mobile applications in the 800 MHz band, which forms part of the ‘Digital Dividend’, will cause interference to TV signals under certain conditions. The new mobile applications (called LTE, Long Term Evolution) use frequencies also used in cable TV networks. This report examines how much interference may occur when providing digital television over cable networks