Evolution of statistical analysis in empirical software engineering research: Current state and steps forward

Software engineering research is evolving and papers are increasingly based on empirical data from a multitude of sources, using statistical tests to determine if and to what degree empirical evidence supports their hypotheses. To investigate the practices and trends of statistical analysis in empirical software engineering (ESE), this paper presents a review of a large pool of papers from top-ranked software engineering journals. First, we manually reviewed 161 papers and in the second phase of our method, we conducted a more extensive semi-automatic classification of papers spanning the years 2001--2015 and 5,196 papers. Results from both review steps was used to: i) identify and analyze the predominant practices in ESE (e.g., using t-test or ANOVA), as well as relevant trends in usage of specific statistical methods (e.g., nonparametric tests and effect size measures) and, ii) develop a conceptual model for a statistical analysis workflow with suggestions on how to apply different statistical methods as well as guidelines to avoid pitfalls. Lastly, we confirm existing claims that current ESE practices lack a standard to report practical significance of results. We illustrate how practical significance can be discussed in terms of both the statistical analysis and in the practitioner's context.Comment: journal submission, 34 pages, 8 figure

The Influence of Situational Involvement on Employees’ Intrinsic Involvement During IS Development

The accelerated pace of digital technology development and adoption and the ensuing digital disruption challenge established business models at many levels, particularly by invalidating traditional value proposition logics. Therefore, processes of technology and information system (IS) adoption and implementation are crucial to organizations striving to survive in complex digitalized environments. In these circumstances, organizations should be aware of and minimize the possibilities of not using IS. The user involvement perspective may help organizations face this issue. Involving users in IS implementation through activities, agreements, and behavior during system development activities (what the literature refers to as situational involvement) may be an effective way to increase user psychological identification with the system, achieving what the literature describes as intrinsic involvement, a state that ultimately helps to increase the adoption rate. Nevertheless, it is still necessary to understand the influence of situational involvement on intrinsic involvement. Thus, the paper explores how situational involvement and intrinsic involvement relate through a fractional factorial experiment with engineering undergraduate students. The resulting model explains 57.79% of intrinsic involvement and supports the importance of the theoretical premise that including users in activities that nurture a sense of responsibility contributes toward system implementation success. To practitioners, the authors suggest that convenient and low-cost hands-on activities may contribute significantly to IS implementation success in organizations. The study also contributes to adoption and diffusion theory by exploring the concept of user involvement, usually recognized as necessary for an IS adoption but not entirely contemplated in the key adoption and diffusion models

Empirical assessment of the effort needed to attack programs protected with client/server code splitting

Context. Code hardening is meant to fight malicious tampering with sensitive code executed on client hosts. Code splitting is a hardening technique that moves selected chunks of code from client to server. Although widely adopted, the effective benefits of code splitting are not fully understood and thoroughly assessed. Objective. The objective of this work is to compare non protected code vs. code splitting protected code, considering two levels of the chunk size parameter, in order to assess the effectiveness of the protection - in terms of both attack time and success rate - and to understand the attack strategy and process used to overcome the protection. Method. We conducted an experiment with master students performing attack tasks on a small application hardened with different levels of protection. Students carried out their task working at the source code level. Results. We observed a statistically significant effect of code splitting on the attack success rate that, on the average, was reduced from 89% with unprotected clear code to 52% with the most effective protection. The protection variant that moved some small-sized code chunks turned out to be more effective than the alternative moving fewer but larger chunks. Different strategies were identified yielding different success rates. Moreover, we discovered that successful attacks exhibited different process w.r.t. failed ones.Conclusions We found empirical evidence of the effect of code splitting, assessed the relative magnitude, and evaluated the influence of the chunk size parameter. Moreover, we extracted the process used to overcome such obfuscation technique

Empirical Standards for Software Engineering Research

Empirical Standards are natural-language models of a scientific community's expectations for a specific kind of study (e.g. a questionnaire survey). The ACM SIGSOFT Paper and Peer Review Quality Initiative generated empirical standards for research methods commonly used in software engineering. These living documents, which should be continuously revised to reflect evolving consensus around research best practices, will improve research quality and make peer review more effective, reliable, transparent and fair.Comment: For the complete standards, supplements and other resources, see https://github.com/acmsigsoft/EmpiricalStandard

Celebrando los 50 años de la revista interamericana de psicologia/interamerican journal of psychology: un análisis de contenido

The Revista Interamericana de Psicologia/Interamerican Journal of Psychology (RIP/IJP) was first published in 1967. During its fifty-year history, the goal of the journal has been to promote collaboration and communication among psychologists on the Americas through the dissemination of theoretical and applied research. The authors reviewed 1,200 articles published in the RIP/IJP between 1967 and 2016 (Volumes 1-50). The articles were coded into seven main content categories, which included type of article, research design, sample characteristics, data collection strategies, data analysis, thematic analysis, and author’s characteristics and affiliations. Results indicated that up to date, 2,688 authors from 30 identified countries have contributed research on a diverse array of psychological topics on primarily three languages: Spanish, English, and Portuguese. Data also suggest these articles are characterized by a broad range of research methodologies, data collection procedures and analysis, and sample characteristics.La Revista Interamericana de Psicología/ /Interamerican Journal of Psychology (RIP/IJP) se publicó por primera vez en el 1967. Durante sus cincuenta años de historia, el objetivo de la revista ha sido promover la colaboración y la comunicación entre los psicólogos de las Américas a través de la difusion de investigaciones teoricas y aplicadas. Los autores revisaron 1.200 artículos publicados en el RIP/IJP entre 1967 y 2016 (volúmenes 1-50). Los artículos se codificaron en siete categorías principales de contenido que incluyeron el tipo de artículo, el diseño de la investigación, las características de la muestra, las estrategias de recopilación de datos, el análisis de datos, el análisis temático y las características y afiliaciones del autor. Los resultados indicaron que hasta la fecha, 2,688 autores de 30 países han contribuido con investigaciones sobre una gran variedad de temas psicológicos en principalmente tres idiomas: español, inglés y portugués. Los datos también sugieren que estos artículos se caracterizan por una amplia gama de metodologías de investigación, procedimientos y análisis de recopilación de datos y características de la muestra.Fil: Torres Fernandez, Ivelisse. New Mexico State University.; Estados UnidosFil: Polanco, Fernando Andrés. Universidad Nacional de San Luis. Facultad de Ciencias Físico Matemáticas y Naturales. Departamento de Informática. Laboratorio Investigación y Desarrollo en Inteligencia Computacional; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - San Luis; ArgentinaFil: Pereira, Steve R.. New Mexico State University.; Estados UnidosFil: Béria, Josiane Sueli. Universidad Nacional de San Luis. Facultad de Psicología. Departamento de Formación Profesional; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - San Luis; ArgentinaFil: Zapico, Martín Gonzalo. Universidad Nacional de San Luis; Argentin

Soft Skills and Software Development: A Reflection from the Software Industry

Psychological theories assert that not everybody is fit for every task, as people have different personality traits and abilities. Often, personality traits are expressed in people’s soft skills. That is, the way people perceive, plan and execute any assigned task is influenced by their set of soft skills. Most of the studies carried out on the human factor in IS concentrate primarily on personality types. Soft skills have been given comparatively little attention by researchers. We review the literature relating to soft skills and the software engineering and information systems domain before describing a study based on 650 job advertisements posted on well-known recruitment sites from a range of geographical locations including, North America, Europe, Asia and Australia. The study makes use of nine defined soft skills to assess the level of demand for each of these skills related to individual job roles within the software industry. This work reports some of the vital statistics from industry about the requirements of soft skills in various roles of software development phases. The work also highlights the variation in the types of skills required for each of the roles. We found that currently although the software industry is paying attention to soft skills up to some extent while hiring but there is a need to further acknowledge the role of these skills in software development. The objective of this paper is to analyze the software industry’s soft skills requirements for various software development positions, such as system analyst, designer, programmer, and tester. We pose two research questions, namely, (1) What soft skills are appropriate to different software development lifecycle roles, and (2) Up to what extend does the software industry consider soft skills when hiring an employee. The study suggests that there is a further need of acknowledgment of the significance of soft skills from employers in software industry

Referencing sources in discussion and conclusion sections

For the past 25 years, the structure of academic research papers has been examined by linguists. In the present study, the functions of references in discussion and conclusion sections of academic research papers were explored using a modified version of moves and steps outlined in Swales (1990). References in 12 disciplines of the annotated Iowa State University Academic Writing Corpus were examined for their frequency. The highest frequency disciplines were further investigated to find out the functions of the references, how these vary within and across disciplines, and what patterns exist in regards to the contextual location of references. Biology stood out with extensive use of references for empirical background, while information systems and engineering were characterized by fewer, mostly “uncited” references. Variation in function was found within disciplines, yet discipline–specific characteristics emerged. The findings could be used in graduate–level writing courses to raise awareness of discipline–specific tendencies. Differences in function were found to exist depending on the level of context taken into account, pointing to a need for specificity in research on functional moves. The results also support the notion that a multi–level, move–step analysis is best suited for structural descriptions of discussion and conclusion sections