Evolution of statistical analysis in empirical software engineering research: Current state and steps forward

Software engineering research is evolving and papers are increasingly based on empirical data from a multitude of sources, using statistical tests to determine if and to what degree empirical evidence supports their hypotheses. To investigate the practices and trends of statistical analysis in empirical software engineering (ESE), this paper presents a review of a large pool of papers from top-ranked software engineering journals. First, we manually reviewed 161 papers and in the second phase of our method, we conducted a more extensive semi-automatic classification of papers spanning the years 2001--2015 and 5,196 papers. Results from both review steps was used to: i) identify and analyze the predominant practices in ESE (e.g., using t-test or ANOVA), as well as relevant trends in usage of specific statistical methods (e.g., nonparametric tests and effect size measures) and, ii) develop a conceptual model for a statistical analysis workflow with suggestions on how to apply different statistical methods as well as guidelines to avoid pitfalls. Lastly, we confirm existing claims that current ESE practices lack a standard to report practical significance of results. We illustrate how practical significance can be discussed in terms of both the statistical analysis and in the practitioner's context.Comment: journal submission, 34 pages, 8 figure

Improving the quality of the personalized electronic program guide

As Digital TV subscribers are offered more and more channels, it is becoming increasingly difficult for them to locate the right programme information at the right time. The personalized Electronic Programme Guide (pEPG) is one solution to this problem; it leverages artificial intelligence and user profiling techniques to learn about the viewing preferences of individual users in order to compile personalized viewing guides that fit their individual preferences. Very often the limited availability of profiling information is a key limiting factor in such personalized recommender systems. For example, it is well known that collaborative filtering approaches suffer significantly from the sparsity problem, which exists because the expected item-overlap between profiles is usually very low. In this article we address the sparsity problem in the Digital TV domain. We propose the use of data mining techniques as a way of supplementing meagre ratings-based profile knowledge with additional item-similarity knowledge that can be automatically discovered by mining user profiles. We argue that this new similarity knowledge can significantly enhance the performance of a recommender system in even the sparsest of profile spaces. Moreover, we provide an extensive evaluation of our approach using two large-scale, state-of-the-art online systems—PTVPlus, a personalized TV listings portal and Físchlár, an online digital video library system

Microelectronics Process Engineering at San Jose State University: A Manufacturing-Oriented Interdisciplinary Degree Program

San Jose State University\u27s new interdisciplinary curriculum in Microelectronics Process Engineering is described. This baccalaureate program emphasizes hands-on thin-film fabrication experience, manufacturing methods such as statistical process control, and fundamentals of materials science and semiconductor device physics. Each course of the core laboratory sequence integrates fabrication knowledge with process engineering and manufacturing methods. The curriculum development process relies on clearly defined and detailed program and course learning objectives. We also briefly discuss our strategy of making process engineering experiences accessible for all engineering students through both Lab Module and Statistics Module series

Software component testing : a standard and the effectiveness of techniques

This portfolio comprises two projects linked by the theme of software component testing, which is also often referred to as module or unit testing. One project covers its standardisation, while the other considers the analysis and evaluation of the application of selected testing techniques to an existing avionics system. The evaluation is based on empirical data obtained from fault reports relating to the avionics system. The standardisation project is based on the development of the BC BSI Software Component Testing Standard and the BCS/BSI Glossary of terms used in software testing, which are both included in the portfolio. The papers included for this project consider both those issues concerned with the adopted development process and the resolution of technical matters concerning the definition of the testing techniques and their associated measures. The test effectiveness project documents a retrospective analysis of an operational avionics system to determine the relative effectiveness of several software component testing techniques. The methodology differs from that used in other test effectiveness experiments in that it considers every possible set of inputs that are required to satisfy a testing technique rather than arbitrarily chosen values from within this set. The three papers present the experimental methodology used, intermediate results from a failure analysis of the studied system, and the test effectiveness results for ten testing techniques, definitions for which were taken from the BCS BSI Software Component Testing Standard. The creation of the two standards has filled a gap in both the national and international software testing standards arenas. Their production required an in-depth knowledge of software component testing techniques, the identification and use of a development process, and the negotiation of the standardisation process at a national level. The knowledge gained during this process has been disseminated by the author in the papers included as part of this portfolio. The investigation of test effectiveness has introduced a new methodology for determining the test effectiveness of software component testing techniques by means of a retrospective analysis and so provided a new set of data that can be added to the body of empirical data on software component testing effectiveness

Project for the analysis of technology transfer Quarterly report, 1 Apr. 1969 - 30 Jun. 1969

Patterns, statistical analyses, and case studies of transfer and utilization of NASA generated technolog

Bio-Based Renewable Additives for Anti-Icing Applications (Phase II)

The performance and impacts of several agro-based anti-icers along with a traditional chloride-based anti-icer (salt brine) were evaluated. A statistical design of experiments (central composite design) was employed for developing anti-icing liquids consisting of cost-competitive chemicals such as agro-based compounds (e.g., Concord grape extract and glycerin), sodium chloride, sodium metasilicate, and sodium formate. The following experimentally obtained parameters were examined as a function of the formulation design: ice-melting capacity at 25°F (−3.9°C), splitting strength of Portland cement mortar samples after 10 freeze-thaw/deicer cycles, corrosion rate of C1010 carbon steel after 24-hour immersion, and impact on asphalt binder stiffness and m-value. One viable formula (“best performer”) was tested for thermal properties by measuring its differential scanning calorimetry (DSC) thermograms, the friction coefficient of asphalt pavement treated by this anti-icing formulation (vs. 23 wt.% NaCl and beet juice blend) at 25°F after being applied at 30 gallons per lane mile (1 hour after simulated trafficking and plowing), and other properties (pH, oxygen demand in COD). Laboratory data shed light on the selection and formulation of innovative agro-based snow- and ice-control chemicals that can significantly reduce the costs of winter maintenance operations

Project for the analysis of technology transfer Quarterly report, 13 Jul. - 12 Oct. 1968

Statistical characteristics of transfer data bank users, and outline of technology transfer and utilization instruction cours

Classifying the Correctness of Generated White-Box Tests: An Exploratory Study

White-box test generator tools rely only on the code under test to select test inputs, and capture the implementation's output as assertions. If there is a fault in the implementation, it could get encoded in the generated tests. Tool evaluations usually measure fault-detection capability using the number of such fault-encoding tests. However, these faults are only detected, if the developer can recognize that the encoded behavior is faulty. We designed an exploratory study to investigate how developers perform in classifying generated white-box test as faulty or correct. We carried out the study in a laboratory setting with 54 graduate students. The tests were generated for two open-source projects with the help of the IntelliTest tool. The performance of the participants were analyzed using binary classification metrics and by coding their observed activities. The results showed that participants incorrectly classified a large number of both fault-encoding and correct tests (with median misclassification rate 33% and 25% respectively). Thus the real fault-detection capability of test generators could be much lower than typically reported, and we suggest to take this human factor into account when evaluating generated white-box tests.Comment: 13 pages, 7 figure

Scientific Excellence in the Forensic Science Community

This Article was prepared as a companion to the Fordham Law Review Reed Symposium on Forensic Expert Testimony, Daubert, and Rule 702, held on October 27, 2017, at Boston College School of Law. The Symposium took place under the sponsorship of the Judicial Conference Advisory Committee on Evidence Rules. For an overview of the Symposium, see Daniel J. Capra, Foreword: Symposium on Forensic Testimony, Daubert, and Rule 702, 86 Fordham L. Rev. 1459 (2018)