Requirements Prioritisation and Retrospective Analysis for Release Planning Process Improvement

The quality of a product can be defined by its ability to satisfy the needs and expectations of its customers. Achieving quality is especially difficult in market-driven situations since the product is released on an open market with numerous potential customers and users with various wishes. The quality of the software product is to a large extent determined by the quality of the requirements engineering (RE) and release planning decisions regarding which requirements that are selected for a product. The goal of this thesis is to enhance software product quality and increase the competitive edge of software organisations by improving release planning decision-making. The thesis is based on empirical research, including both qualitative and quantitative research approaches. The research contains a qualitative survey of RE challenges in market-driven organisations based on interviews with practitioners. The survey provided increased understanding of RE challenges in the software industry and gave input to the continued research. Among the challenging issues, one was selected for further investigation due to its high relevance to the practitioners: requirements prioritisation and release planning decision-making. Requirements prioritisation techniques were evaluated through experiments, suggesting that ordinal scale techniques based on grouping and ranking may be valuable to practitioners. Finally, a retrospective method called PARSEQ (Post-release Analysis of Requirements SElection Quality) is introduced and tested in three case studies. The method aims at evaluating prior releases and finding improvement proposals for release planning decision-making in future release projects. The method was found valuable by all participants and relevant improvement proposals were discovered in all cases

Linking Long-Term Product Planning and Requirements Prioritization to Customer Value Creation

Creating value for different customer segments is essential to the business of a company. Thus, software product development companies' ability to implement the most valuable requirements in their products has been seen as critical. The literature offers requirements prioritization methods for selecting requirements, but their suitability for solving practical challenges is not clear. The state of the practice in long-term product planning and requirements prioritization, and the practical challenges involved is not thoroughly analysed. Therefore, the connection between the selection of product features and customer value creation is also an area that needs more investigation. This thesis investigates the current state of long-term product planning and requirements prioritization, and their linkages to customer value creation in market-driven software product development. The results are based on the experience gathered from 7 Finnish software product development companies that had recognized the importance to improve their long-term product planning and requirements prioritization practices. To gain a deep understanding in real product development context, we conducted longitudinal case studies using an action research approach. In addition, we evaluated the suitability of requirements prioritization methods in product development projects. The thesis provides a systematic analysis of long-term product planning and requirements prioritization activities and challenges involved in the market-driven software product development companies. According to our results product planning activities are at a low-level, focusing on the individual features of individual products in short term. Practitioners have challenges in drawing different viewpoints together into product planning. Both integrating business and customer viewpoints to the project level as well as integrating the understanding about technical limitations and possibilities to product level planning was difficult in practice. The requirements prioritization methods did not solve the practical challenges. On the basis of our findings we also propose a set of practices that support the link from long-term product planning and requirements prioritization to customer value creation. The essence of the suggested practices is 1) in the shift of planning focus from individual product features towards understanding the processes of customers and users and in describing the solutions from the customer viewpoint as a whole and 2) in the cross-functional effort to be able to combine the three main viewpoints of customer value creation (company's own business, customers and users, and implementation).Arvon luominen eri asiakassegmenteille on elintärkeää yrityksen liiketoiminnalle. Ohjelmistotuoteyrityksen kyvykkyys toteuttaa tuotteisiinsa asiakkaiden näkökulmasta katsoen arvokkaimmat ominaisuudet onkin siksi nähty kriittisenä. Kirjallisuudessa esitellään erilaisia priorisointimenetelmiä tuoteominaisuuksien valintaan, mutta niiden soveltuvuudesta käytännön ohjelmistotyöhön ei juuri tiedetä. Myöskään pitkäntähtäimen suunnittelun ja vaatimusten priorisoinnin nykytilaa sekä niihin liittyviä käytännön haasteita ei ole syvällisesti analysoitu. Siksi yhteyttä ominaisuusvalintojen ja asiakasarvon luomisen välillä onkin tärkeää tutkia lisää. Tässä tutkimuksessa selvitettiin, miten pitkäntähtäimen tuotesuunnittelu ja vaatimusten priorisointi tukevat arvon luontia asiakkaille ja käyttäjille markkinaohjautuvassa ohjelmistotuotekehityksessä. Tulokset perustuvat seitsemästä suomalaisesta ohjelmistotuoteyrityksestä kerättyihin kokemuksiin. Saadaksemme tuotesuunnittelusta ja ominaisuusvalintojen tekemisestä niin pitkällä kuin lyhyelläkin tähtäimellä syvällisen ymmärryksen, toteutimme pitkittäisiä tapaustutkimuksia käyttäen toimintatutkimuslähestymistapaa. Lisäksi kokeilimme vaatimusten priorisointimenetelmiä todellisessa tuotekehitysympäristössä. Tutkimus esittelee systemaattisen nykytila-analyysin ohjelmistoyritysten tuotesuunnittelun käytännöistä ja haasteista. Tulokset paljastavat, että ohjelmistotuotteita suunnitellaan pirstaleisesti, lyhyellä aikajänteellä ja yksittäisen tuotteen ominaisuuksiin kerralla keskittyen. Yritysten työntekijöillä on vaikeuksia käytännössä yhdistää arvonluonnin kannalta tärkeitä näkökulmia. Sekä liiketoiminta- ja asiakasnäkökulman tuominen tuotekehitysprojektien päätöksentekoon että toisaalta teknisten rajoitteiden ja mahdollisuuksien huomioiminen jo tuotesuunnittelun aiemmissa vaiheissa ovat erityisen haastavia. Tutkimuksen tuloksena syntyi käytäntöjä, joiden avulla asiakasarvon luomista voisi tukea tuotesuunnittelussa. Näiden käytäntöjen keskiössä ovat 1) suunnittelun fokuksen siirtäminen yksittäisistä tuoteominaisuuksista asiakkaan ja loppukäyttäjän prosessien ymmärtämiseen, sekä ratkaisujen kuvaaminen asiakkaan näkökulmasta kokonaisuuksina, sekä 2) yli organisaatiorajojen ulottuva työskentely, jonka avulla voidaan paremmin yhdistää kolme tärkeintä asiakasarvon luonnin näkökulmaa (yrityksen oma liiketoiminta, asiakkaat ja käyttäjät, sekä tuotteen toteuttaminen)

Increasing workload on simulated remotely piloted system interaction and task completion – gamers versus non-gamers

With the current high rate of development and deployment of Remotely Piloted Aerial Systems (RPAS) for both commercial and military sectors globally, it is key to understand the implications this technology has on current and future RPAS operators and the consequential effect on licensing, training and performance measurement. This thesis investigates aspects of training and potential objective performance measurement of RPAS operators, this is carried out by reviewing current literature relating to RPAS and associated human factors thus a gap analysis was undertaken and a set of experiments/evaluations were devised to provide important new insights. Attention is drawn to the type of skill set required for future RPAS operations. A factor has been to understand whether a regular computer games player displays differing simulator interaction, in this case information gathering and analysis patterns, to that of someone with limited to no computer games experience. To achieve the aims of the research experimentation had to be carried which required the development of an appropriate simulator followed by the inclusion of a case study and the creation of bespoke performance data analysis software, SimPACT. Although performance differentials have been observed through action it was hoped to be able to identify performance differential characteristics through the means of evaluating the use of disparate physical data sets; the research, in fact, identified no significant difference between data set use and it must be concluded that any pre-action performance differential cannot be measured, at least not with the equipment available. However computer gamers, rather than having differing information acquisition strategies, have differing and more effective information retention and processing pathways likely to have been developed through continuous gaming which can be applied to any game-type environment and, potentially, any type of interactive task. These results have been proven to be statistically viable and observable. This research has contributed to the understanding of human performance measurement within the RPAS sector, including the addition of new data processing software, as well as provide new evidence relating to difference within human data gathering and processing between groups of differing experiences

Incident Prioritisation for Intrusion Response Systems

The landscape of security threats continues to evolve, with attacks becoming more serious and the number of vulnerabilities rising. To manage these threats, many security studies have been undertaken in recent years, mainly focusing on improving detection, prevention and response efficiency. Although there are security tools such as antivirus software and firewalls available to counter them, Intrusion Detection Systems and similar tools such as Intrusion Prevention Systems are still one of the most popular approaches. There are hundreds of published works related to intrusion detection that aim to increase the efficiency and reliability of detection, prevention and response systems. Whilst intrusion detection system technologies have advanced, there are still areas available to explore, particularly with respect to the process of selecting appropriate responses. Supporting a variety of response options, such as proactive, reactive and passive responses, enables security analysts to select the most appropriate response in different contexts. In view of that, a methodical approach that identifies important incidents as opposed to trivial ones is first needed. However, with thousands of incidents identified every day, relying upon manual processes to identify their importance and urgency is complicated, difficult, error-prone and time-consuming, and so prioritising them automatically would help security analysts to focus only on the most critical ones. The existing approaches to incident prioritisation provide various ways to prioritise incidents, but less attention has been given to adopting them into an automated response system. Although some studies have realised the advantages of prioritisation, they released no further studies showing they had continued to investigate the effectiveness of the process. This study concerns enhancing the incident prioritisation scheme to identify critical incidents based upon their criticality and urgency, in order to facilitate an autonomous mode for the response selection process in Intrusion Response Systems. To achieve this aim, this study proposed a novel framework which combines models and strategies identified from the comprehensive literature review. A model to estimate the level of risks of incidents is established, named the Risk Index Model (RIM). With different levels of risk, the Response Strategy Model (RSM) dynamically maps incidents into different types of response, with serious incidents being mapped to active responses in order to minimise their impact, while incidents with less impact have passive responses. The combination of these models provides a seamless way to map incidents automatically; however, it needs to be evaluated in terms of its effectiveness and performances. To demonstrate the results, an evaluation study with four stages was undertaken; these stages were a feasibility study of the RIM, comparison studies with industrial standards such as Common Vulnerabilities Scoring System (CVSS) and Snort, an examination of the effect of different strategies in the rating and ranking process, and a test of the effectiveness and performance of the Response Strategy Model (RSM). With promising results being gathered, a proof-of-concept study was conducted to demonstrate the framework using a live traffic network simulation with online assessment mode via the Security Incident Prioritisation Module (SIPM); this study was used to investigate its effectiveness and practicality. Through the results gathered, this study has demonstrated that the prioritisation process can feasibly be used to facilitate the response selection process in Intrusion Response Systems. The main contribution of this study is to have proposed, designed, evaluated and simulated a framework to support the incident prioritisation process for Intrusion Response Systems.Ministry of Higher Education in Malaysia and University of Malay

Impact estimation: IT priority decisions

Given resource constraints, prioritization is a fundamental process within systems engineering to decide what to implement. However, there is little guidance about this process and existing IT prioritization methods have several problems, including failing to adequately cater for stakeholder value. In response to these issues, this research proposes an extension to an existing prioritization method, Impact Estimation (IE) to create Value Impact Estimation (VIE). VIE extends IE to cater for multiple stakeholder viewpoints and to move towards better capture of explicit stakeholder value. The use of metrics offers VIE the means of expressing stakeholder value that relates directly to real world data and so is informative to stakeholders and decision makers. Having been derived from prioritization factors found in the literature, stakeholder value has been developed into a multi-dimensional, composite concept, associated with other fundamental system concepts: objectives, requirements, designs, increment plans, increment deliverables and system contexts. VIE supports the prioritization process by showing where the stakeholder value resides for the proposed system changes. The prioritization method was proven to work by exposing it to three live projects, which served as case studies to this research. The use of the extended prioritization method was seen as very beneficial. Based on the three case studies, it is possible to say that the method produces two major benefits: the calculation of the stakeholder value to cost ratios (a form of ROI) and the system understanding gained through creating the VIE table

Agile Ways of Working: A Team Maturity Perspective

With the agile approach to managing software development projects comes an increased dependability on well functioning teams, since many of the practices are built on teamwork. The objective of this study was to investigate if, and how, team development from a group psychological perspective is related to some work practices of agile teams. Data were collected from 34 agile teams (200 individuals) from six software development organizations and one university in both Brazil and Sweden using the Group Development Questionnaire (Scale IV) and the Perceptive Agile Measurement (PAM). The result indicates a strong correlation between levels of group maturity and the two agile practices \emph{iterative development} and \emph{retrospectives}. We, therefore, conclude that agile teams at different group development stages adopt parts of team agility differently, thus confirming previous studies but with more data and by investigating concrete and applied agile practices. We thereby add evidence to the hypothesis that an agile implementation and management of agile projects need to be adapted to the group maturity levels of the agile teams

Prioritisation of requests, bugs and enhancements pertaining to apps for remedial actions. Towards solving the problem of which app concerns to address initially for app developers

Useful app reviews contain information related to the bugs reported by the app’s end-users along with the requests or enhancements (i.e., suggestions for improvement) pertaining to the app. App developers expend exhaustive manual efforts towards the identification of numerous useful reviews from a vast pool of reviews and converting such useful reviews into actionable knowledge by means of prioritisation. By doing so, app developers can resolve the critical bugs and simultaneously address the prominent requests or enhancements in short intervals of apps’ maintenance and evolution cycles. That said, the manual efforts towards the identification and prioritisation of useful reviews have limitations. The most common limitations are: high cognitive load required to perform manual analysis, lack of scalability associated with limited human resources to process voluminous reviews, extensive time requirements and error-proneness related to the manual efforts. While prior work from the app domain have proposed prioritisation approaches to convert reviews pertaining to an app into actionable knowledge, these studies have limitations and lack benchmarking of the prioritisation performance. Thus, the problem to prioritise numerous useful reviews still persists. In this study, initially, we conducted a systematic mapping study of the requirements prioritisation domain to explore the knowledge on prioritisation that exists and seek inspiration from the eminent empirical studies to solve the problem related to the prioritisation of numerous useful reviews. Findings of the systematic mapping study inspired us to develop automated approaches for filtering useful reviews, and then to facilitate their subsequent prioritisation. To filter useful reviews, this work developed six variants of the Multinomial Naïve Bayes method. Next, to prioritise the order in which useful reviews should be addressed, we proposed a group-based prioritisation method which initially classified the useful reviews into specific groups using an automatically generated taxonomy, and later prioritised these reviews using a multi-criteria heuristic function. Subsequently, we developed an individual prioritisation method that directly prioritised the useful reviews after filtering using the same multi-criteria heuristic function. Some of the findings of the conducted systematic mapping study not only provided the necessary inspiration towards the development of automated filtering and prioritisation approaches but also revealed crucial dimensions such as accuracy and time that could be utilised to benchmark the performance of a prioritisation method. With regards to the proposed automated filtering approach, we observed that the performance of the Multinomial Naïve Bayes variants varied based on their algorithmic structure and the nature of labelled reviews (i.e., balanced or imbalanced) that were made available for training purposes. The outcome related to the automated taxonomy generation approach for classifying useful review into specific groups showed a substantial match with the manual taxonomy generated from domain knowledge. Finally, we validated the performance of the group-based prioritisation and individual prioritisation methods, where we found that the performance of the individual prioritisation method was superior to that of the group-based prioritisation method when outcomes were assessed for the accuracy and time dimensions. In addition, we performed a full-scale evaluation of the individual prioritisation method which showed promising results. Given the outcomes, it is anticipated that our individual prioritisation method could assist app developers in filtering and prioritising numerous useful reviews to support app maintenance and evolution cycles. Beyond app reviews, the utility of our proposed prioritisation solution can be evaluated on software repositories tracking bugs and requests such as Jira, GitHub and so on