Conducting usability testing in agile software development environment

Abstract. This Master’s thesis conducted a qualitative approached case study with triangulation to action research for usability testing in Agile software development. First, there was done literature review to create theoretical basis for research. After literature review, the research method was constructed from usability testing methods and qualitative research tools. When research method was explained, the implementation of the research method was opened to tell how the created research method was used. Explained implementation was followed by presenting the findings. Then discussion conducted cohesion between literature, research method, implementation, and findings. Finally, conclusions summarized the main points of the thesis. Literature review brought up several usability testing methods. Research method formed to use UCD, interview, SUS, and TAP from the usability testing methods that were found. Implementation conducted eight questions for semi structured interview, five tasks for second sprint’s TAP and four tasks for third sprint’s SUS. It was identified that TAP and SUS can be used in the same usability testing session. Moreover, there were found nine hypotheses to be tested. Eight of those hypotheses were successfully tested. It was also found that action research requires adaption during the research and Agile generates changes in the plans successfully. Usability testing should be done by using at least one usability testing method and taking end user into test sessions as soon as possible. Semi structured interview’s benefit is to have more open conversation type moment with the test user. TAP achieves descriptive analysis considering how test user experiences usability of the application. SUS offers general usability level of the application in the numeric scale. Improving steps for usability testing can be seen from type of the need for the different usability responses. Another way to identify improvement is to test a new usability testing method. One way to bring usability testing in Agile software development is to conduct usability testing at the end of every sprint

Data cleaning techniques for software engineering data sets

Data quality is an important issue which has been addressed and recognised in research communities such as data warehousing, data mining and information systems. It has been agreed that poor data quality will impact the quality of results of analyses and that it will therefore impact on decisions made on the basis of these results. Empirical software engineering has neglected the issue of data quality to some extent. This fact poses the question of how researchers in empirical software engineering can trust their results without addressing the quality of the analysed data. One widely accepted definition for data quality describes it as `fitness for purpose', and the issue of poor data quality can be addressed by either introducing preventative measures or by applying means to cope with data quality issues. The research presented in this thesis addresses the latter with the special focus on noise handling. Three noise handling techniques, which utilise decision trees, are proposed for application to software engineering data sets. Each technique represents a noise handling approach: robust filtering, where training and test sets are the same; predictive filtering, where training and test sets are different; and filtering and polish, where noisy instances are corrected. The techniques were first evaluated in two different investigations by applying them to a large real world software engineering data set. In the first investigation the techniques' ability to improve predictive accuracy in differing noise levels was tested. All three techniques improved predictive accuracy in comparison to the do-nothing approach. The filtering and polish was the most successful technique in improving predictive accuracy. The second investigation utilising the large real world software engineering data set tested the techniques' ability to identify instances with implausible values. These instances were flagged for the purpose of evaluation before applying the three techniques. Robust filtering and predictive filtering decreased the number of instances with implausible values, but substantially decreased the size of the data set too. The filtering and polish technique actually increased the number of implausible values, but it did not reduce the size of the data set. Since the data set contained historical software project data, it was not possible to know the real extent of noise detected. This led to the production of simulated software engineering data sets, which were modelled on the real data set used in the previous evaluations to ensure domain specific characteristics. These simulated versions of the data set were then injected with noise, such that the real extent of the noise was known. After the noise injection the three noise handling techniques were applied to allow evaluation. This procedure of simulating software engineering data sets combined the incorporation of domain specific characteristics of the real world with the control over the simulated data. This is seen as a special strength of this evaluation approach. The results of the evaluation of the simulation showed that none of the techniques performed well. Robust filtering and filtering and polish performed very poorly, and based on the results of this evaluation they would not be recommended for the task of noise reduction. The predictive filtering technique was the best performing technique in this evaluation, but it did not perform significantly well either. An exhaustive systematic literature review has been carried out investigating to what extent the empirical software engineering community has considered data quality. The findings showed that the issue of data quality has been largely neglected by the empirical software engineering community. The work in this thesis highlights an important gap in empirical software engineering. It provided clarification and distinctions of the terms noise and outliers. Noise and outliers are overlapping, but they are fundamentally different. Since noise and outliers are often treated the same in noise handling techniques, a clarification of the two terms was necessary. To investigate the capabilities of noise handling techniques a single investigation was deemed as insufficient. The reasons for this are that the distinction between noise and outliers is not trivial, and that the investigated noise cleaning techniques are derived from traditional noise handling techniques where noise and outliers are combined. Therefore three investigations were undertaken to assess the effectiveness of the three presented noise handling techniques. Each investigation should be seen as a part of a multi-pronged approach. This thesis also highlights possible shortcomings of current automated noise handling techniques. The poor performance of the three techniques led to the conclusion that noise handling should be integrated into a data cleaning process where the input of domain knowledge and the replicability of the data cleaning process are ensured.EThOS - Electronic Theses Online ServiceGBUnited Kingdo