Security Verification and Validation by Software SMEs: Theory versus Practice

To improve software engineering practice it is essential to observe the socio-technical realities that surround software development within an industrial context. There is a lack of empirical knowledge of security verification and validation practice within an SME context. When coupled with the recognised importance, and inherent complexities, of such practice, it appears fundamentally sound to understand the faced socio-technical realities to ensure continued process improvement and improved technology adoption and research guidance. Within this research-in-progress paper we highlight the importance of obtaining such an understanding

Towards a Model of Testers' Cognitive Processes: Software Testing as a Problem Solving Approach

Software testing is a complex, intellectual activity based (at least) on analysis, reasoning, decision making, abstraction and collaboration performed in a highly demanding environment. Naturally, it uses and allocates multiple cognitive resources in software testers. However, while a cognitive psychology perspective is increasingly used in the general software engineering literature, it has yet to find its place in software testing. To the best of our knowledge, no theory of software testers' cognitive processes exists. Here, we take the first step towards such a theory by presenting a cognitive model of software testing based on how problem solving is conceptualized in cognitive psychology. Our approach is to instantiate a general problem solving process for the specific problem of creating test cases. We then propose an experiment for testing our cognitive test design model. The experiment makes use of verbal protocol analysis to understand the mechanisms by which human testers choose, design, implement and evaluate test cases. An initial evaluation was then performed with five software engineering master students as subjects. The results support a problem solving-based model of test design for capturing testers' cognitive processes.Comment: (v3) minor issues fixed, Accepted and presented in the IEEE International Workshop on Human and Social Aspects of Software Quality (HASQ 2020

Towards a hybrid testing process unifying exploratory testing and scripted testing

CONTEXT Given the current state of the art in research, practitioners are faced with the challenge of choosing scripted testing (ST) or exploratory testing (ET). OBJECTIVE This study aims at systematically incorporating strengths of ET and ST in a hybrid testing process to overcome the weaknesses of each. METHOD We utilized systematic review and practitioner interviews to identify strengths and weaknesses of ET and ST. Strengths of ET were mapped to weaknesses of ST and vice versa. Noblit and Hare's lines-ofargument method was used for data analysis. The results of the mapping were used as input to codesign a hybrid process with experienced practitioners. RESULTS We found a clear need to create a hybrid process as follows: (i) both ST and ET provide strengths and weaknesses, and these depend on some particular conditions, which prevents preference of one approach to another; and (ii) the mapping showed that it is possible to address the weaknesses in one process by the strengths of the other in a hybrid form. With the input from literature and industry experts, a flexible and iterative hybrid process was designed. CONCLUSIONS Practitioners can clearly benefit from using a hybrid process given the mapping of advantages and disadvantage

Towards a Model of Testers\u27 Cognitive Processes: Software Testing as a Problem Solving Approach

Software testing is a complex, intellectual activity based (at least) on analysis, reasoning, decision making, abstraction and collaboration performed in a highly demanding environment. Naturally, it uses and allocates multiple cognitive resources in software testers. However, while a cognitive psychology perspective is increasingly used in the general software engineering literature, it has yet to find its place in software testing. To the best of our knowledge, no theory of software testers\u27 cognitive processes exists. Here, we take the first step towards such a theory by presenting a cognitive model of software testing based on how problem solving is conceptualized in cognitive psychology. Our approach is to instantiate a general problem solving process for the specific problem of creating test cases. We then propose an experiment for testing our cognitive test design model. The experiment makes use of verbal protocol analysis to understand the mechanisms by which human testers choose, design, implement and evaluate test cases. An initial evaluation was then performed with five software engineering master students as subjects. The results support a problem solving-based model of test design for capturing testers\u27 cognitive processes

Implementing Test Automation with Selenium WebDriver

Many software programs, such as applications for designing, modeling, simulating, and analyzing systems, are now commonly available as web-based applications. The testing of such sophisticated web applications is highly challenging and can be extremely tedious and error-prone if done manually. Recently automation tools have become increasingly used for testing web-based applications, as they minimize human involvement and repetitive work. For this problem report project, we have built and implemented an automation testing framework for web applications. The project specifically uses a tool called Selenium WebDriver, which has been used to develop the testing framework. By using this framework, testers may quickly and effectively write their test cases. The benefits of Selenium WebDriver include that it does not require in-depth research and training by testers, and due to the framework\u27s ability to take screenshots, it provides a useful way for developers to study their code. The framework relies on the Chrome web browser, along with Java running in Eclipse, to provide a user-friendly interface for constructing and running test suites. To validate the testing framework, we performed a case study involving NanoHub (nanoHUB.org), which is a well-known platform that provides valuable resources for those involved in nanotechnology research and education. NanoHub serves as an open-access repository for a wide range of tools, simulations, and information related to nanoscale science and engineering, and it is designed particularly to model and simulate electronic systems and nanoscale phenomena. Testing a website such as NanoHub.org typically encompasses a blend of functional testing, usability testing, and performance testing. Based on the results of this testing, several observations are made about the testing framework in general, and its application to NanoHub in particular. The comprehensive testing approach documented in this report is aimed at ensuring the platform functions as intended, provides a user-friendly experience, and delivers optimal performance. This testing is particularly crucial when dealing with tools and simulations related to electronic systems

Sapienz: Multi-objective automated testing for android applications

We introduce Sapienz, an approach to Android testing that uses multi-objective search-based testing to automatically explore and optimise test sequences, minimising length, while simultaneously maximising coverage and fault revelation. Sapienz combines random fuzzing, systematic and search-based exploration, exploiting seeding and multi-level instrumentation. Sapienz significantly outperforms (with large effect size) both the state-of-the-art technique Dynodroid and the widely-used tool, Android Monkey, in 7/10 experiments for coverage, 7/10 for fault detection and 10/10 for fault-revealing sequence length. When applied to the top 1, 000 Google Play apps, Sapienz found 558 unique, previously unknown crashes. So far we have managed to make contact with the developers of 27 crashing apps. Of these, 14 have confirmed that the crashes are caused by real faults. Of those 14, six already have developer-confirmed fixes

Pembangkit Kasus Uji Berbasis Model Pada Antarmuka (GUI) Aplikasi Android

Antarmuka pengguna grafis merupakan salah satu bagian penting dari perangkat lunak karena pengguna berinteraksi dengan perangkat lunak melalui widget seperti button, text field, dan image button yang terdapat pada antarmuka pengguna grafis. Membangun antarmuka pengguna grafis memerlukan source code yang cukup banyak sesuai dengan kebutuhan antarmuka yang dibangun, sehingga menyebabkan rentan terjadi kesalahan pada antarmuka pengguna grafis. Salah satu kesalahan pada antarmuka pengguna grafis adalah incorrect state of widgets. Kesalahan incorrect state of widgets menyebabkan keadaan dan respon dari widget berbeda dari hasilnya yang diharapkan. Kesalahan ini juga akan memberikan dampak yang negatif bagi pengguna aplikasi. Oleh karena itu, pengembang perlu melakukan salah satu tahapan penting dalam siklus hidup perangkat lunak yaitu pengujian. Untuk melakukan pengujian, diperlukan adanya test case. Penelitian ini bertujuan untuk membangun tool yang dapat membangkitkan test case secara otomatis untuk mengatasi kesalahan incorrect state of widgets pada antarmuka pengguna grafis. Adapun metode yang diusulkan adalah model-based testing. Hasil test case yang diperoleh melalui tool akan dievaluasi dengan menggunakan metode manual exploratory testing. Hasil dari evaluasi yang dilakukan menyatakan bahwa hasil test case dari tool lebih baik untuk digunakan dalam melakukan pengujian antarmuka pengguna grafis karena test case yang diperoleh hampir mencakup seluruh bagian widget dan condition pada antarmuka aplikasi

Automaatiotestaaminen web-sovelluksen kehityksessä

Tiivistelmä. Digitalisoituminen ohjaa finanssialan sovellusten kehitysmenetelmiä kohti ketteriä toimintamalleja, joissa jatkuva muutos on normi. Testaus on keskeinen osa sovelluskehitystä, mutta sen on mukauduttava jatkuvaan kehitykseen ja siksi sen toteuttamiseen tarvitaan automatisoituja ratkaisuja. Automaation avulla voidaan nopeuttaa prosesseja ja vähentää manuaalista työtä, laajentaa suoritettavien tehtävien määrää ja kehittää testausprosesseja ja -työkaluja vastaamaan kehityksen vaatimuksia. Tässä diplomityössä esitellään testaustoimintamalleja ja -työkaluja ketterään web-sovelluskehitykseen pankkitoimialalla. Tutkimuksessa tarkastellaan pankkijärjestelmään kehitetyn sovelluksen kehitysprosessia ja etsitään ratkaisuja kehitystiimin kohtaamiin testaushaasteisiin. Tutkielman aikana kokeiltiin käyttöliittymätestien ajamista ennen koodikatselmointia, mikä lisäsi testien ajoa ja kehittäjien kiinnostusta testien ylläpitoon. Tutkielmasta löydettyjä ratkaisuja voidaan soveltaa yrityksissä ja kehitystiimeissä kehitysmallin parantamiseksi ja sulauttamiseksi testaustoimintamalleihin. Testaustoimintamallien sulauttaminen kehitysprosessiin voi parantaa tiimin työskentelymallia ja ehkäistä tehtävien siiloutumista kehittäjien välillä. Oikean järjestyksen varmistaminen kehitysprosessissa voi edesauttaa kehitystiimin tuottavuutta ja vähentää myöhään havaittavien ongelmien riskiä. Tämä tutkielma tarjoaa suuntaviivoja web-sovelluskehityksen kehitysmallille.Automated testing in web application development. Abstract. Digitization is driving the development methods of financial industry applications towards agile operating models, where constant change is the norm. Testing is a key part of application development, but it needs to adapt to continuous development, which is why automated solutions are needed. Automation can speed up processes, reduce manual work, expand the number of tasks performed, and develop testing processes and tools to meet development requirements. This thesis presents testing methods and tools for agile web application development in the banking industry. The study examines the development process of an application developed for a banking system and seeks solutions to testing challenges faced by the development team. During the study, the running of GUI tests before code review was tested, which increased test runs and developers’ interest in maintaining tests. The solutions found in the thesis can be applied in companies and development teams to improve the development model and integrate it with testing models. Integrating testing models into the development process can improve the team’s working model and prevent silos between developers. Ensuring the correct order in the development process can help increase the productivity of the development team and reduce the risk of late-detected issues. This thesis provides guidelines for the development model of web application development