Parallel, Cross-Platform Unit Testing for Real-Time Embedded Systems

Embedded systems are used in a wide variety of applications (e.g., automotive, agricultural, home security, industrial, medical, military, and aerospace) due to their small size, low-energy consumption, and the ability to control real-time peripheral devices precisely. These systems, however, are different from each other in many aspects: processors, memory size, develop applications/OS, hardware interfaces, and software loading methods. Unit testing is a fundamental part of software development and the lowest level of software testing, as it tests individual or groups of functions, methods, and classes, to increase confidence that the developed software satisfies both software specifications and user requirements. Although hundreds of unit testing frameworks exist, none of them address the diverse properties of real-time embedded platforms. This inspires us to introduce XEUnit, a cross-platform unit testing framework for real-time embedded systems. XEUnit provides scalability to the framework by supporting parallel execution on multiple embedded platforms simultaneously. To address the time constraints in real-time embedded systems, we evaluate the impact of runtime overhead from traditional instrumentation through a case study of time-sensitive algorithms. Then, we introduce iterative instrumentation, which is a code coverage technique without runtime overhead, along with a case study demonstrating the effectiveness of this technique. Although iterative instrumentation can measure code coverage effectively in time-sensitive applications, the total execution cost of this approach is much higher than traditional instrumentation due to the execution of multiple variants of the system under test. This leads to scalability and performance issues especially in large applications. To solve these issues, there are two approaches we use: reducing the number of variants and executing them simultaneously. To reduce the number of variants, we present cluster iterative instrumentation, a graph clustering technique that can reduce the number of nodes in a control flow graph resulting in lower execution time. We also provide a case study of node coverage of control software to show the efficiency of cluster iterative instrumentation compared to iterative instrumentation. In addition to reducing the number of variants, the other method is to execute multiple variants at the same time. Because all executions are independent from each other, we can use parallel execution on multiple embedded platforms. Thus, we design and implement a parallel unit testing framework for real-time embedded system along with a case study comparing the execution times from different numbers of embedded platforms (executing nodes). Our final contribution is a cross-platform unit testing framework using the concepts of runtime adapters and a runtime protocol that enables testers to run code across different embedded platforms. We also demonstrate the effectiveness of this framework by testing black-box test cases on seven different embedded platforms. Overall, our results indicate that cluster iterative instrumentation with parallel unit testing can address the scalability and performance issues, and the case studies demonstrate that XEUnit can effectively test the same code on a variety of embedded platforms

A Platform-Based Software Design Methodology for Embedded Control Systems: An Agile Toolkit

A discrete control system, with stringent hardware constraints, is effectively an embedded real-time system and hence requires a rigorous methodology to develop the software involved. The development methodology proposed in this paper adapts agile principles and patterns to support the building of embedded control systems, focusing on the issues relating to a system's constraints and safety. Strong unit testing, to ensure correctness, including the satisfaction of timing constraints, is the foundation of the proposed methodology. A platform-based design approach is used to balance costs and time-to-market in relation to performance and functionality constraints. It is concluded that the proposed methodology significantly reduces design time and costs, as well as leading to better software modularity and reliability

Design Space Exploration in Cyber-Physical Systems

Cyber physical systems (CPS) integrate a variety of engineering areas such as control, mechanical and computer engineering in a holistic design effort. While interdependencies between the different disciplines are key attributes of CPS design science, little is known about the impact of design decisions of the cyber part on the overall system qualities. To investigate these interdependencies, this paper proposes a simulation-based Design Space Exploration (DSE) framework that considers detailed cyber system parameters such as cache size, bus width, and voltage levels in addition to physical and control parameters of the CPS. We propose an exploration algorithm that surfs the parameter configurations in the cyber physical sub-systems, in order to approximate the Pareto-optimal design points with regards to the trade-os among the design objectives, such as energy consumption and control stability. We apply the proposed framework to a network control system for an inverted-pendulum application. The presented holistic evaluation of the identified Pareto-points reveals the presence of non-trivial trade-os, which are imposed by the control, physical, and detailed cyber parameters. For instance the identified energy and control optimal design points comprise configurations with a wide range of CPU speeds, sample times and cache configuration following non-trivial zig-zag patterns. The proposed framework could identify and manage those trade-os and, as a result, is an imperative rst step to automate the search for superior CSP configurations

A Survey of Techniques For Improving Energy Efficiency in Embedded Computing Systems

Recent technological advances have greatly improved the performance and features of embedded systems. With the number of just mobile devices now reaching nearly equal to the population of earth, embedded systems have truly become ubiquitous. These trends, however, have also made the task of managing their power consumption extremely challenging. In recent years, several techniques have been proposed to address this issue. In this paper, we survey the techniques for managing power consumption of embedded systems. We discuss the need of power management and provide a classification of the techniques on several important parameters to highlight their similarities and differences. This paper is intended to help the researchers and application-developers in gaining insights into the working of power management techniques and designing even more efficient high-performance embedded systems of tomorrow

SmartUnit: Empirical Evaluations for Automated Unit Testing of Embedded Software in Industry

In this paper, we aim at the automated unit coverage-based testing for embedded software. To achieve the goal, by analyzing the industrial requirements and our previous work on automated unit testing tool CAUT, we rebuild a new tool, SmartUnit, to solve the engineering requirements that take place in our partner companies. SmartUnit is a dynamic symbolic execution implementation, which supports statement, branch, boundary value and MC/DC coverage. SmartUnit has been used to test more than one million lines of code in real projects. For confidentiality motives, we select three in-house real projects for the empirical evaluations. We also carry out our evaluations on two open source database projects, SQLite and PostgreSQL, to test the scalability of our tool since the scale of the embedded software project is mostly not large, 5K-50K lines of code on average. From our experimental results, in general, more than 90% of functions in commercial embedded software achieve 100% statement, branch, MC/DC coverage, more than 80% of functions in SQLite achieve 100% MC/DC coverage, and more than 60% of functions in PostgreSQL achieve 100% MC/DC coverage. Moreover, SmartUnit is able to find the runtime exceptions at the unit testing level. We also have reported exceptions like array index out of bounds and divided-by-zero in SQLite. Furthermore, we analyze the reasons of low coverage in automated unit testing in our setting and give a survey on the situation of manual unit testing with respect to automated unit testing in industry.Comment: In Proceedings of 40th International Conference on Software Engineering: Software Engineering in Practice Track, Gothenburg, Sweden, May 27-June 3, 2018 (ICSE-SEIP '18), 10 page

Opt: A Domain Specific Language for Non-linear Least Squares Optimization in Graphics and Imaging

Many graphics and vision problems can be expressed as non-linear least squares optimizations of objective functions over visual data, such as images and meshes. The mathematical descriptions of these functions are extremely concise, but their implementation in real code is tedious, especially when optimized for real-time performance on modern GPUs in interactive applications. In this work, we propose a new language, Opt (available under http://optlang.org), for writing these objective functions over image- or graph-structured unknowns concisely and at a high level. Our compiler automatically transforms these specifications into state-of-the-art GPU solvers based on Gauss-Newton or Levenberg-Marquardt methods. Opt can generate different variations of the solver, so users can easily explore tradeoffs in numerical precision, matrix-free methods, and solver approaches. In our results, we implement a variety of real-world graphics and vision applications. Their energy functions are expressible in tens of lines of code, and produce highly-optimized GPU solver implementations. These solver have performance competitive with the best published hand-tuned, application-specific GPU solvers, and orders of magnitude beyond a general-purpose auto-generated solver

A software controlled voltage tuning system using multi-purpose ring oscillators

This paper presents a novel software driven voltage tuning method that utilises multi-purpose Ring Oscillators (ROs) to provide process variation and environment sensitive energy reductions. The proposed technique enables voltage tuning based on the observed frequency of the ROs, taken as a representation of the device speed and used to estimate a safe minimum operating voltage at a given core frequency. A conservative linear relationship between RO frequency and silicon speed is used to approximate the critical path of the processor. Using a multi-purpose RO not specifically implemented for critical path characterisation is a unique approach to voltage tuning. The parameters governing the relationship between RO and silicon speed are obtained through the testing of a sample of processors from different wafer regions. These parameters can then be used on all devices of that model. The tuning method and software control framework is demonstrated on a sample of XMOS XS1-U8A-64 embedded microprocessors, yielding a dynamic power saving of up to 25% with no performance reduction and no negative impact on the real-time constraints of the embedded software running on the processor