Software industry experiments: a systematic literature review

There is no specialized survey of experiments conducted in the software industry. Goal: Identify the major features of software industry experiments, such as time distribution, independent and dependent variables, subject types, design types and challenges. Method: Systematic literature review, taking the form of a scoping study. Results: We have identified 10 experiments and five quasi-experiments up to July 2012. Most were run as of 2003. The main features of these studies are that they test technologies related to quality and management and analyse outcomes related to effectiveness and effort. Most experiments have a factorial design. The major challenges faced by experimenters are to minimize the cost of running the experiment for the company and to schedule the experiment so as not to interfere with production processes

Component-based software engineering: a quantitative approach

Dissertação apresentada para a obtenção do Grau de Doutor em Informática pela Universidade Nova de Lisboa, Faculdade de Ciências e TecnologiaBackground: Often, claims in Component-Based Development (CBD) are only supported by qualitative expert opinion, rather than by quantitative data. This contrasts with the normal practice in other sciences, where a sound experimental validation of claims is standard practice. Experimental Software Engineering (ESE) aims to bridge this gap. Unfortunately, it is common to find experimental validation efforts that are hard to replicate and compare, to build up the body of knowledge in CBD. Objectives: In this dissertation our goals are (i) to contribute to evolution of ESE, in what concerns the replicability and comparability of experimental work, and (ii) to apply our proposals to CBD, thus contributing to its deeper and sounder understanding. Techniques: We propose a process model for ESE, aligned with current experimental best practices, and combine this model with a measurement technique called Ontology-Driven Measurement (ODM). ODM is aimed at improving the state of practice in metrics definition and collection, by making metrics definitions formal and executable,without sacrificing their usability. ODM uses standard technologies that can be well adapted to current integrated development environments. Results: Our contributions include the definition and preliminary validation of a process model for ESE and the proposal of ODM for supporting metrics definition and collection in the context of CBD. We use both the process model and ODM to perform a series experimental works in CBD, including the cross-validation of a component metrics set for JavaBeans, a case study on the influence of practitioners expertise in a sub-process of component development (component code inspections), and an observational study on reusability patterns of pluggable components (Eclipse plug-ins). These experimental works implied proposing, adapting, or selecting adequate ontologies, as well as the formal definition of metrics upon each of those ontologies. Limitations: Although our experimental work covers a variety of component models and, orthogonally, both process and product, the plethora of opportunities for using our quantitative approach to CBD is far from exhausted. Conclusions: The main contribution of this dissertation is the illustration, through practical examples, of how we can combine our experimental process model with ODM to support the experimental validation of claims in the context of CBD, in a repeatable and comparable way. In addition, the techniques proposed in this dissertation are generic and can be applied to other software development paradigms.Departamento de Informática of the Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa (FCT/UNL); Centro de Informática e Tecnologias da Informação of the FCT/UNL; Fundação para a Ciência e Tecnologia through the STACOS project(POSI/CHS/48875/2002); The Experimental Software Engineering Network (ESERNET);Association Internationale pour les Technologies Objets (AITO); Association forComputing Machinery (ACM

Information Technology and Systems - I Systems Analysis and Design: Should We Be Researching What We Teach?

A guiding premise of academic scholarship is that knowledge gained from first-hand research experience is disseminated to students via the classroom. However, that valuable connection is lost when professors are not researching what they teach. In this paper, we explore issues of mismatch between teaching and research in the Information Systems (IS) discipline. Specifically, while systems analysis and design (SA&D) is an integral topic in IS curricula, this topic is the research specialty of few IS professors. This situation is reflected by the low number of research publications in this area; particularly in the leading mainstream IS journals. We characterize the gap between teaching and research in SA&D, offer possible explanations for this gap, suggest avenues to better understand and enhance SA&D research via the design science paradigm, list a number of areas in SA&D in which there is ample need and opportunity for high quality research, and show through an example how a research mindset can be incorporated in a graduate level SA&D course

Towards Diagram Understanding: A Pilot Study Measuring Cognitive Workload Through Eye-Tracking

We investigate model understanding, in particular , how the quality of the UML diagram layout impacts cognitive load. We hypothesize that this w ill have a significant impact on the structure and effectiveness of engineers’ communication. In previous work, we have studied task performance measurements and subjective assessments; here, we also investigate behavioral indicators such as fixation and pupillary dilation. We use such indicators to explore diagram understanding- and reading strategies and how such strategies are impacted, e.g. by diagram type and expertise level. In the pilot eye-tracking experiment run so far, we have only examined a small number of participants (n=4), so our results are preliminary in nature and do not afford far reaching conclusions. They do, however, corroborate findings from earlier experiments, for example, showing that layout quality indeed matters and improves understanding. Our results also give rise to a number of new hypotheses about diagram understanding strategies that we are investigating in an ongoing data acquisition campaign