Requirements Engineering that Balances Agility of Teams and System-level Information Needs at Scale

Context: Motivated by their success in software development, large-scale systems development companies are increasingly adopting agile methods and their practices. Such companies need to accommodate different development cycles of hardware and software and are usually subject to regulation and safety concerns. Also, for such companies, requirements engineering is an essential activity that involves upfront and detailed analysis which can be at odds with agile development methods. Objective: The overall aim of this thesis is to investigate the challenges and solution candidates of performing effective requirements engineering in an agile environment, based on empirical evidence. Illustrated with studies on safety and system-level information needs, we explore RE challenges and solutions in large-scale agile development, both in general and from the teams’ perspectives. Method: To meet our aim, we performed a secondary study and a series of empirical studies based on case studies. We collected qualitative data using interviews, focus groups and workshops to derive challenges and potential solutions from industry. Findings: Our findings show that there are numerous challenges of conducting requirements engineering in agile development especially where systems development is concerned. The challenges discovered sprout from an integration problem of working with agile methods while relying on established plan-driven processes for the overall system. We highlight the communication challenge of crossing the boundary of agile methods and system-level (or plan-driven) development, which also proves the coexistence of both methods. Conclusions: Our results highlight the painful areas of requirements engineering in agile development and propose solutions that can be explored further. This thesis contributes to future research, by establishing a holistic map of challenges and candidate solutions that can be further developed to make RE more efficient within agile environments

Requirements Engineering Challenges of Supporting Agile Teams in System Development

Context: Agile methods have attracted many companies due to their reported benefits of short time-to-market and improved quality outputs. In the systems development context, additional constraints apply e.g. as a result of scale or parallel development of hardware and software. Traditionally, stage-gate processes with a focus on up-front requirements analysis are common in large- scale systems engineering. These processes clash with the companies’ desire to become more agile.Objective: The aim of this thesis is to discover challenges that new requirements engineering approaches should address to enable agile system devel- opment at scale (RE4Agile). With a focus on value and building system understanding, we explore these challenges from the perspective of the agile development teams.Method: To meet our aim, we conducted a series of empirical studies based on case studies, and a secondary review to explore the problem domain while deriving challenges and potential solutions from industry and literature respectively.Findings: Our findings show that there are numerous challenges of conducting requirements engineering in agile development especially where systems development is concerned. These challenges relate to user value and overall system understanding. However, there are some cross-cutting concerns, e.g safety- critical development, that have generated much interest both from practitioners and academicians at large.Conclusions: The challenges discovered sprout from an integration problem of working with agile methods while using the already existing processes as well. However, solution candidates exist and our future research aims to validate some of the solution candidates in the view of deriving new RE approaches. This thesis contributes to such future research, by establishing a holistic map of challenges that allows to assess whether a given solution is beneficial in the larger context or whether it over-optimizes only one area

Software Startups -- A Research Agenda

Software startup companies develop innovative, software-intensive products within limited time frames and with few resources, searching for sustainable and scalable business models. Software startups are quite distinct from traditional mature software companies, but also from micro-, small-, and medium-sized enterprises, introducing new challenges relevant for software engineering research. This paper's research agenda focuses on software engineering in startups, identifying, in particular, 70+ research questions in the areas of supporting startup engineering activities, startup evolution models and patterns, ecosystems and innovation hubs, human aspects in software startups, applying startup concepts in non-startup environments, and methodologies and theories for startup research. We connect and motivate this research agenda with past studies in software startup research, while pointing out possible future directions. While all authors of this research agenda have their main background in Software Engineering or Computer Science, their interest in software startups broadens the perspective to the challenges, but also to the opportunities that emerge from multi-disciplinary research. Our audience is therefore primarily software engineering researchers, even though we aim at stimulating collaborations and research that crosses disciplinary boundaries. We believe that with this research agenda we cover a wide spectrum of the software startup industry current needs

The link between transformational and servant leadership in DevOps-oriented organizations

DevOps is a set of agile and lean practices and principles in the context of software product development aiming to decrease mean time-to-market and mean time-to-recover-from-failure through a shift in organizational mindset-skillset-toolset. There is literature to suggest that adopting DevOps has been challenging in practice and that a particular leadership style is necessary to lead DevOps adoption. There are studies to suggest that DevOps leadership is mainly related to transformational leadership characteristics. In this research, a mixed methods approach is used. Initially, semi-structured interviews are conducted with 30 EMEA (Europe, Middle-East and Africa) agile and lean practitioners holding more than 10 years of practitioner experience (81%) from the private and public sectors. The contribution also includes an analysis and evaluation of a survey completed by 250 participants of which 93% works in Europe and Middle East and 76% has held previous leadership positions. By looking to recent literature we identified agile, lean and DevOps practices and principles. In addition, we identify benefits and inhibitors to DevOps adoption and its leadership. Our results suggest that deep rooted organizational culture and lack of DevOps definition clarity are usually considered impediments to DevOps adoption followed by poor communication and collaboration. Our results also show that certain DevOps adoption leadership characteristics are relevant to transformational leadership and servant leadership. The research results also indicate that the DevOps adoption leadership role is linked to certain metrics

A case study of agile software development for large-scale safety-critical systems projects

This study explores the introduction of agile software development within an avionics company engaged in safety-critical system engineering. There is increasing pressure throughout the software industry for development efforts to adopt agile software development in order to respond more rapidly to changing requirements and make more frequent deliveries of systems to customers for review and integration. This pressure is also being experienced in safety-critical industries, where release cycles on typically large and complex systems may run to several years on projects spanning decades. However, safety-critical system developments are normally highly regulated, which may constrain the adoption of agile software development or require adaptation of selected methods or practices. To investigate this potential conflict, we conducted a series of interviews with practitioners in the company, exploring their experiences of adopting agile software development and the challenges encountered. The study also explores the opportunities for altering the existing software process in the company to better fit agile software development to the constraints of software development for safety-critical systems. We conclude by identifying immediate future research directions to better align the tempo of software development for safety-critical systems and agile software development

An agile information flow consolidator for delivery of quality software projects: technological perspective from a South African start-up

In today’s knowledge-based economy, modern organisations understand the importance of technology in their quest to be considered global leaders. South African markets like others worldwide are regularly flooded with the latest technology trends which can complicate the acquisition, use, management and maintenance of software. To achieve a competitive edge, companies tend to leverage agile methods with the best possible combination of innovative supporting tools as a key differentiator. Software technology firms are in this light faced with determining how to leverage technology and efficient development processes for them to consistently deliver quality software projects and solutions to their customer base. Previous studies have discussed the importance of software development processes from a project management perspective. African academia has immensely contributed in terms of software development and project management research which has focused on modern frameworks, methodologies as well as project management techniques. While the current research continues with this tradition by presenting the pertinence of modern agile methodologies, it additionally further describes modern agile development processes tailored in a sub-Saharan context. The study also aims novelty by showing how innovative sometimes disruptive technology tools can contribute to producing African software solutions to African problems. To this end, the thesis contains an experimental case study where a web portal is prototyped to assist firms with the management of agile project management and engineering related activities. Literature review, semi-structure interviews as well as direct observations from the industry use case are used as data sources. Underpinned by an Activity Theory analytical framework, the qualitative data is analysed by leveraging content and thematic oriented techniques. This study aims to contribute to software engineering as well as the information systems body of knowledge in general. The research hence ambitions to propose a practical framework to promote the delivery of quality software projects and products. For this thesis, such a framework was designed around an information system which helps organizations better manage agile project management and engineering related activities.Information SciencePh. D. (Information Systems

Teams in agile software development: Design principles and examination of human factors

In response to new customer requirements, market dynamics, mergers, and technological innovation, modern software development organizations are adopting agile software development (ASD). Yet, the simple adoption of agile methods such as Scrum or eXtreme programming does not automatically result in a very agile team. While we understand the introduction and adoption of ASD from a methodical perspective, we have yet to explore design principles that guide methodical extensions of ASD, and we need to learn more about the human factors that influence software development teams. This thesis presents four studies. Studies 1 and 2 investigate the methodical extension of ASD by identifying design principles from secondary data. Study 1 extends ASD with processes and practices from user-centered design. Study 2 investigates early activities that precede development activities. The thesis also investigates human factors of agile software development in studies 3 and 4. Study 3 compares teams along their extents of agility in order to identify influential factors using a multicase study design. Study 4 tests the effects of emotional contagion in virtual software development teams using a large dataset from an open source software repository. Thus, this thesis makes two primary contributions. First, it develops design principles for methodical extensions of ASD; second, it contributes to the human factors that influence software development teams. Managers also receive guidance on the improvement of ASD in their organization

Describing software developers affectiveness through Markov chain models

In this paper, we present an analysis of more than 500K comments from open-source repositories of software systems. Our aim is to empirically determine how developers interact with each other under certain psychological conditions generated by politeness, sentiment and emotion expressed within developers' comments. Developers involved in an open-source projects do not usually know each other; they mainly communicate through mailing lists, chat rooms, and tools such as issue tracking systems. The way in which they communicate affects the development process and the productivity of the people involved in the project. We evaluated politeness, sentiment and emotions of comments posted by developers and studied the communication flow to understand how they interacted in the presence of impolite and negative comments (and vice versa).Our analysis shows that when in presence of impolite or negative comments, the probability of the next comment being impolite or negative is 14% and 25%, respectively; anger however, has a probability of 40% of being followed by a further anger comment. The result could help managers take control the development phases of a system, since social aspects can seriously affect a developer's productivity. In a distributed environment this may have a particular resonance.Engineering and Physical Sciences Research Council (EPSRC

On the real world practice of Behaviour Driven Development

Surveys of industry practice over the last decade suggest that Behaviour Driven Development is a popular Agile practice. For example, 19% of respondents to the 14th State of Agile annual survey reported using BDD, placing it in the top 13 practices reported. As well as potential benefits, the adoption of BDD necessarily involves an additional cost of writing and maintaining Gherkin features and scenarios, and (if used for acceptance testing,) the associated step functions. Yet there is a lack of published literature exploring how BDD is used in practice and the challenges experienced by real world software development efforts. This gap is significant because without understanding current real world practice, it is hard to identify opportunities to address and mitigate challenges. In order to address this research gap concerning the challenges of using BDD, this thesis reports on a research project which explored: (a) the challenges of applying agile and undertaking requirements engineering in a real world context; (b) the challenges of applying BDD specifically and (c) the application of BDD in open-source projects to understand challenges in this different context. For this purpose, we progressively conducted two case studies, two series of interviews, four iterations of action research, and an empirical study. The first case study was conducted in an avionics company to discover the challenges of using an agile process in a large scale safety critical project environment. Since requirements management was found to be one of the biggest challenges during the case study, we decided to investigate BDD because of its reputation for requirements management. The second case study was conducted in the company with an aim to discover the challenges of using BDD in real life. The case study was complemented with an empirical study of the practice of BDD in open source projects, taking a study sample from the GitHub open source collaboration site. As a result of this Ph.D research, we were able to discover: (i) challenges of using an agile process in a large scale safety-critical organisation, (ii) current state of BDD in practice, (iii) technical limitations of Gherkin (i.e., the language for writing requirements in BDD), (iv) challenges of using BDD in a real project, (v) bad smells in the Gherkin specifications of open source projects on GitHub. We also presented a brief comparison between the theoretical description of BDD and BDD in practice. This research, therefore, presents the results of lessons learned from BDD in practice, and serves as a guide for software practitioners planning on using BDD in their projects