Reducing the Barrier to Entry of Complex Robotic Software: a MoveIt! Case Study

Developing robot agnostic software frameworks involves synthesizing the disparate fields of robotic theory and software engineering while simultaneously accounting for a large variability in hardware designs and control paradigms. As the capabilities of robotic software frameworks increase, the setup difficulty and learning curve for new users also increase. If the entry barriers for configuring and using the software on robots is too high, even the most powerful of frameworks are useless. A growing need exists in robotic software engineering to aid users in getting started with, and customizing, the software framework as necessary for particular robotic applications. In this paper a case study is presented for the best practices found for lowering the barrier of entry in the MoveIt! framework, an open-source tool for mobile manipulation in ROS, that allows users to 1) quickly get basic motion planning functionality with minimal initial setup, 2) automate its configuration and optimization, and 3) easily customize its components. A graphical interface that assists the user in configuring MoveIt! is the cornerstone of our approach, coupled with the use of an existing standardized robot model for input, automatically generated robot-specific configuration files, and a plugin-based architecture for extensibility. These best practices are summarized into a set of barrier to entry design principles applicable to other robotic software. The approaches for lowering the entry barrier are evaluated by usage statistics, a user survey, and compared against our design objectives for their effectiveness to users

Cross-platform verification framework for embedded systems

Many innovations in the automotive sector involve complex electronics and embedded software systems. Testing techniques are one of the key methodologies for detecting faults in such embedded systems.In this paper, a novel cross-platform verification framework including automated test-case generation by model checking is introduced. Comparing the execution behavior of a program instance running on a certain platform to the execution behavior of the same program running on a different platform we denote cross-platform verification. The framework supports various types of coverage criteria. It turned out that end-to-end testing is of high importance due to defects occurring on the actual target platform for the first time.Additionally, formal verification can be applied for checking requirements resulting from the specification using the same model generation mechanism that is used for test data generation. Due to a novel self-assessment mechanism, the confidence into the formal models is increased significantly.We provide a case study for the Motorola embedded controller HCS12 that is heavily used by the automotive industry. We perform structural tests on industrial code patterns using a wide-spread industrial compiler. Using our technique, we found two severe compiler defects that have been corrected in subsequent releases

LittleDarwin: a Feature-Rich and Extensible Mutation Testing Framework for Large and Complex Java Systems

Mutation testing is a well-studied method for increasing the quality of a test suite. We designed LittleDarwin as a mutation testing framework able to cope with large and complex Java software systems, while still being easily extensible with new experimental components. LittleDarwin addresses two existing problems in the domain of mutation testing: having a tool able to work within an industrial setting, and yet, be open to extension for cutting edge techniques provided by academia. LittleDarwin already offers higher-order mutation, null type mutants, mutant sampling, manual mutation, and mutant subsumption analysis. There is no tool today available with all these features that is able to work with typical industrial software systems.Comment: Pre-proceedings of the 7th IPM International Conference on Fundamentals of Software Engineerin

Tools and Algorithms for the Construction and Analysis of Systems

This open access two-volume set constitutes the proceedings of the 27th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS 2021, which was held during March 27 – April 1, 2021, as part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2021. The conference was planned to take place in Luxembourg and changed to an online format due to the COVID-19 pandemic. The total of 41 full papers presented in the proceedings was carefully reviewed and selected from 141 submissions. The volume also contains 7 tool papers; 6 Tool Demo papers, 9 SV-Comp Competition Papers. The papers are organized in topical sections as follows: Part I: Game Theory; SMT Verification; Probabilities; Timed Systems; Neural Networks; Analysis of Network Communication. Part II: Verification Techniques (not SMT); Case Studies; Proof Generation/Validation; Tool Papers; Tool Demo Papers; SV-Comp Tool Competition Papers

Pairwise-Korat : automated testing using Korat in an industrial setting

textIn this report, we present an algorithm for testing applications which takes structurally complex test inputs. The algorithm, Pairwise-Korat, adopts Korat -- an algorithm for constraint-based generation of structurally complex test inputs. Korat takes (1) an imperative predicate which specifies the desired structural integrity constraints and (2) a finitization which bounds the desired test inputs size. Korat performs a systematic search to generate all test inputs (within the bounds) for which the predicate returns true. We present how to generate test inputs in Korat and how to execute test inputs in parallel. The inputs that Korat generates enable bounded-exhaustive testing that checks the code under test exhaustively for all inputs within the given bounds. We also describe a novel methodology for reducing the number of equivalent inputs that Korat generates. Our development of test input generation and the methodology for reducing equivalent inputs are motivated by testing applications developed at eBay. The experimental results show that the Pairwise-Korat achieves great performance in finding defects and increasing test coverage and the algorithm outperforms current manual solutions adopted at the company.Electrical and Computer Engineerin

Addressing aspect interactions in an industrial setting: experiences, problems and solutions

Aspect oriented programming (AOP) introduces new and powerful modularization constructs. The aspect module is used to encapsulate crosscutting concerns, which otherwise would remain tangled and scattered. The idea of encapsulating crosscutting concerns rapidly expanded to earlier phases in the development cycle, including requirement analysis (aspect oriented requirement engineering, AORE) and design (aspect oriented modeling, AOM). The overall application of aspect orientation concepts is known as aspect oriented software development (AOSD). AOP is not yet a mainstream practice. Particularly AOSD is still in its early stages. This is reflected in the lack of reports of full development cycles using aspect oriented approaches, especially using industrial case studies. Furthermore, the power of aspects comes at the price of new challenges, one of them is that systems built using aspects are more difficult to understand. The crosscutting nature of aspects allows them to alter the behavior of many other modules. As a result, aspects may interact in unintended and unanticipated ways. This problem is known as aspect interactions. In this work we deal with the aspect interaction problem in the context of an industrial domain: slots machines. We perform a complete development cycle of the slot machine software. This is, to the best of our knowledge, the first complete industrial case of study of aspect orientation. Through this experience we discovered the limitations with regard to aspect interactions, of some emblematic aspect oriented approaches for requirement engineering, design and implementation. The contribution of this work is threefold. Firstly, we contribute with the evaluation and extensions to some of AORE and AOM approaches, in order to provide explicit support for aspect interactions in requirement analysis and design phases. We also evaluate the implementation of interactions using a static and a dynamic AOP language, and propose an AspectJ extension that copes with aspect interactions. Secondly, this work is the first report of a complete aspect oriented development cycle of an industrial case study. Thirdly, this work provides a complex case study that presents several business logic crosscutting concerns, which in turn exhibit numerous aspect interactions, that serves as a challenging test bed for upcoming AOSD approaches.Facultad de Informátic

Building an open-source system test generation tool: lessons learned and empirical analyses with EvoMaster

Research in software testing often involves the development of software prototypes. Like any piece of software, there are challenges in the development, use and verification of such tools. However, some challenges are rather specific to this problem domain. For example, often these tools are developed by PhD students straight out of bachelor/master degrees, possibly lacking any industrial experience in software development. Prototype tools are used to carry out empirical studies, possibly studying different parameters of novel designed algorithms. Software scaffolding is needed to run large sets of experiments efficiently. Furthermore, when using AI-based techniques like evolutionary algorithms, care needs to be taken to deal with their randomness, which further complicates their verification. The aforementioned represent some of the challenges we have identified for this domain. In this paper, we report on our experience in building the open-source EvoMaster tool, which aims at system-level test case generation for enterprise applications. Many of the challenges we faced would be common to any researcher needing to build software testing tool prototypes. Therefore, one goal is that our shared experience here will boost the research community, by providing concrete solutions to many development challenges in the building of such kind of research prototypes. Ultimately, this will lead to increase the impact of scientific research on industrial practice.publishedVersio

Testing of xtUML Models across Auto-Reflexive Software Architecture

Application of MDA in the software development enables a synchronization of the system models and corresponding source files used for the building of the executable version of a software system. Because of often use of manual modifications of some parts of code without equivalent changes in connected models, there is no guarantee that the output of the process of building of the target application will be consistent with the relevant design and implementation models. Possibility of generating of the source files from the models is a necessity, but not a sufficient condition in the process of development and modification of software systems synchronously with the changes in all related models.  More safe approach is building the target application with the use of an automated building process with nested steps for consistency verifications of all critical models and related source files and the usage of model compilers. This article describes the method and tools for extending the software process of building the target system using special files with specification of dependencies between models and source files. Such dependencies represent the core of the critical knowledge, and it is possible to make this knowledge an integral part of the proposed new software architecture