Towards a Tool-based Development Methodology for Pervasive Computing Applications

Despite much progress, developing a pervasive computing application remains a challenge because of a lack of conceptual frameworks and supporting tools. This challenge involves coping with heterogeneous devices, overcoming the intricacies of distributed systems technologies, working out an architecture for the application, encoding it in a program, writing specific code to test the application, and finally deploying it. This paper presents a design language and a tool suite covering the development life-cycle of a pervasive computing application. The design language allows to define a taxonomy of area-specific building-blocks, abstracting over their heterogeneity. This language also includes a layer to define the architecture of an application, following an architectural pattern commonly used in the pervasive computing domain. Our underlying methodology assigns roles to the stakeholders, providing separation of concerns. Our tool suite includes a compiler that takes design artifacts written in our language as input and generates a programming framework that supports the subsequent development stages, namely implementation, testing, and deployment. Our methodology has been applied on a wide spectrum of areas. Based on these experiments, we assess our approach through three criteria: expressiveness, usability, and productivity

API para procedimentos de teste em sistemas embutidos

The proposed case study takes a satellite control application as system under test and entails extending a simple simulator for the system component this application interacts with. Atop of these shall be developed test procedures to study whether there are practical limitations of using simple procedures to test complex interactions between the satellite control application and its simulated environment. The objective of the proposed work is to define a test frontend API that enables simple test procedures while providing all the means required to test complex machine-to-machine (M2M) interactions, having the system under test (SUT) hard real-time characteristics.O caso de estudo proposto toma uma aplicação de controlo de um satélite como um sistema a ser testado e implica a extensão de um simulador simples para o componente do sistema com o qual esta aplicação interage. Em cima destes devem ser desenvolvidos procedimentos de teste para estudar se há limitações práticas ao usar procedimentos simples para testar interações complexas entre a aplicação de controlo do satélite e o ambiente simulado. O objetivo do trabalho proposto é definir uma API de teste que possibilite procedimentos de teste simples e forneça todos os meios necessários para testar interações complexas máquina a máquina, tendo o sistema sob teste características de tempo real.Mestrado em Engenharia Eletrónica e Telecomunicaçõe

Scripted GUI Testing of Android Apps: A Study on Diffusion, Evolution and Fragility

Background. Evidence suggests that mobile applications are not thoroughly tested as their desktop counterparts. In particular GUI testing is generally limited. Like web-based applications, mobile apps suffer from GUI test fragility, i.e. GUI test classes failing due to minor modifications in the GUI, without the application functionalities being altered. Aims. The objective of our study is to examine the diffusion of GUI testing on Android, and the amount of changes required to keep test classes up to date, and in particular the changes due to GUI test fragility. We define metrics to characterize the modifications and evolution of test classes and test methods, and proxies to estimate fragility-induced changes. Method. To perform our experiments, we selected six widely used open-source tools for scripted GUI testing of mobile applications previously described in the literature. We have mined the repositories on GitHub that used those tools, and computed our set of metrics. Results. We found that none of the considered GUI testing frameworks achieved a major diffusion among the open-source Android projects available on GitHub. For projects with GUI tests, we found that test suites have to be modified often, specifically 5\%-10\% of developers' modified LOCs belong to tests, and that a relevant portion (60\% on average) of such modifications are induced by fragility. Conclusions. Fragility of GUI test classes constitute a relevant concern, possibly being an obstacle for developers to adopt automated scripted GUI tests. This first evaluation and measure of fragility of Android scripted GUI testing can constitute a benchmark for developers, and the basis for the definition of a taxonomy of fragility causes, and actionable guidelines to mitigate the issue.Comment: PROMISE'17 Conference, Best Paper Awar

Automated testing with Wireless Communication in the digitalised industry : A case study of Mirka Oy

Advanced automation technologies are changing the dynamics of the process and manufacturing industries. Product development processes are becoming smarter with the application of intelligent solutions and automated testing. The industry 4.0 concept of centralized control for industrial devices results in a rapid increase in the demand for the industrial Internet of Things (IoT) and cordless machines. Wireless communication protocols are integral to the functioning of such devices. This thesis work is performed with Mirka Oy during the development process of a smart industrial cordless tool. Various available short-range wireless communication protocols are studied to find out the best possible solution to match the product requirements. Besides, an automated testing platform is developed to verify and validate the functional description of the devices. All the stages, starting from the types of embedded system testing, device test requirements, test case designing leading to a comprehensive testing platform are explained. Results generated by the automated platform are analysed, which shows that all the test execution is successful. The successful implantation of this automated testing platform would significantly increase the efficiency of the development and testing process. Moreover, this dissertation highlights further development in terms of the application of the Artificial Intelligence (AI) and Machine learning (ML) technique for smarter testing processes and increase the overall performance of the testing framework

Development of a self-diagnostic system integrated into a cyber-physical system

CONTROLAR provides Bosch with an intelligent functional testing machine used to test the correct functioning of the car radios produced. During this process, the radios are submitted to several tests, raising the problem of how the machine detects errors in several radios consecutively, making it impossible to know if the device has a problem since it has no module to see if it works correctly. This article arises from the need to find a solution to solve this problem, which was to develop a self-diagnostic system that will ensure the reliability and integrity of the cyber-physical system, passing a detailed state of the art. The development of this system was based on the design of an architecture that combines the KDT methodology with a DSL to manage and configure the tests to integrate the self-diagnostic test system into a CPS. A total of 28 test cases were performed to cover all its functionalities. The results show that all test cases passed. Therefore, the system meets all the proposed objectives.This article is a result of the project POCI-01-0247-FEDER-040130, supported by Operational Program for Competitiveness and Internationalization (COMPETE 2020), under the PORTUGAL 2020 Partnership Agreement, through the European Regional Development Fund (ERDF)

Enhancing the test and evaluation process: implementing agile development, test automation, and model-based systems engineering concepts

2020 Fall.Includes bibliographical references.With the growing complexity of modern systems, traditional testing methods are falling short. Test documentation suites used to verify the software for these types of large, complex systems can become bloated and unclear, leading to extremely long execution times and confusing, unmanageable test procedures. Additionally, the complexity of these systems can prevent the rapid understanding of complicated system concepts and behaviors, which is a necessary part of keeping up with the demands of modern testing efforts. Opportunities for optimization and innovation exist within the Test and Evaluation (T&E) domain, evidenced by the emergence of automated testing frameworks and iterative testing methodologies. Further opportunities lie with the directed expansion and application of related concepts such as Model-Based Systems Engineering (MBSE). This dissertation documents the development and implementation of three methods of enhancing the T&E field when applied to a real-world project. First, the development methodology of the system was transitioned from Waterfall to Agile, providing a more responsive approach when creating new features. Second, the Test Automation Framework (TAF) was developed, enabling the automatic execution of test procedures. Third, a method of test documentation using the Systems Modeling Language (SysML) was created, adopting concepts from MBSE to standardize the planning and analysis of test procedures. This dissertation provides the results of applying the three concepts to the development process of an airborne Electronic Warfare Management System (EWMS), which interfaces with onboard and offboard aircraft systems to receive and process the threat environment, providing the pilot or crew with a response solution for the protection of the aircraft. This system is representative of a traditional, long-term aerospace project that has been constantly upgraded over its lifetime. Over a two-year period, this new process produced a number of qualitative and quantitative results, including improving the quality and organization of the test documentation suite, reducing the minimum time to execute the test procedures, enabling the earlier identification of defects, and increasing the overall quality of the system under test. The application of these concepts generated many lessons learned, which are also provided. Transitioning a project's development methodology, modernizing the test approach, and introducing a new system of test documentation may provide significant benefits to the development of a system, but these types of process changes must be weighed against the needs of the project. This dissertation provides details of the effort to improve the effectiveness of the T&E process on an example project, as a framework for possible implementation on similar systems