Control-Theoretical Perspective in Feedback-Based Systems Testing

Self-Adaptive Systems (SAS) and Cyber-Physical Systems (CPS) have received significant attention in recent computer engineering research. This is due to their ability to improve the level of autonomy of engineering artefacts. In both cases, this autonomy increase is achieved through feedback. Feedback is the iteration of sens- ing and actuation to respectively acquire knowledge about the current state of said artefacts and steer them toward a desired state or behaviour. In this thesis we dis- cuss the challenges that the introduction of feedback poses on the verification and validation process for such systems, more specifically, on their testing. We highlight three types of new challenges with respect to traditional software testing: alteration of testing input and output definition, and intertwining of components with different nature. Said challenges affect the ways we can define different elements of the test- ing process: coverage criteria, testing set-ups, test-case generation strategies, and oracles in the testing process. This thesis consists of a collection of three papers and contributes to the definition of each of the mentioned testing elements. In terms of coverage criteria for SAS, Paper I proposes the casting of the testing problem, to a semi-infinite optimisation problem. This allows to leverage the Scenario Theory from the field of robust control, and provide a worst-case probabilistic bound on a given performance metric of the system under test. For what concerns the definition of testing set-ups for control-based CPS, Paper II investigates the implications of the use of different abstractions (i.e., the use of implemented or emulated compo- nents) on the significance of the testing. The paper provides evidence that confutes the common assumption present in previous literature on the existence of a hierar- chy among commonly used testing set-ups. Finally, regarding the test-case gener- ation and oracle definition, Paper III defines the problem of stress testing control- based CPS software. We contribute to the generation and identification of stress test cases for such software by proposing a novel test case parametrisation. Leveraging the proposed parametrisation we define metamorphic relations on the expected be- haviour of the system under test. We use said relations for the development of stress testing approach and sanity checks on the testing results

On Provably Correct Decision-Making for Automated Driving

The introduction of driving automation in road vehicles can potentially reduce road traffic crashes and significantly improve road safety. Automation in road vehicles also brings several other benefits such as the possibility to provide independent mobility for people who cannot and/or should not drive. Many different hardware and software components (e.g. sensing, decision-making, actuation, and control) interact to solve the autonomous driving task. Correctness of such automated driving systems is crucial as incorrect behaviour may have catastrophic consequences. Autonomous vehicles operate in complex and dynamic environments, which requires decision-making and planning at different levels. The aim of such decision-making components in these systems is to make safe decisions at all times. The challenge of safety verification of these systems is crucial for the commercial deployment of full autonomy in vehicles. Testing for safety is expensive, impractical, and can never guarantee the absence of errors. In contrast, formal methods, which are techniques that use rigorous mathematical models to build hardware and software systems can provide a mathematical proof of the correctness of the system. The focus of this thesis is to address some of the challenges in the safety verification of decision-making in automated driving systems. A central question here is how to establish formal verification as an efficient tool for automated driving software development.A key finding is the need for an integrated formal approach to prove correctness and to provide a complete safety argument. This thesis provides insights into how three different formal verification approaches, namely supervisory control theory, model checking, and deductive verification differ in their application to automated driving and identifies the challenges associated with each method. It identifies the need for the introduction of more rigour in the requirement refinement process and presents one possible solution by using a formal model-based safety analysis approach. To address challenges in the manual modelling process, a possible solution by automatically learning formal models directly from code is proposed

Autonomous agile teams: Challenges and future directions for research

According to the principles articulated in the agile manifesto, motivated and empowered software developers relying on technical excellence and simple designs, create business value by delivering working software to users at regular short intervals. These principles have spawned many practices. At the core of these practices is the idea of autonomous, self-managing, or self-organizing teams whose members work at a pace that sustains their creativity and productivity. This article summarizes the main challenges faced when implementing autonomous teams and the topics and research questions that future research should address

Supporting decision making process with "Ideal" software agents: what do business executives want?

According to Simon’s (1977) decision making theory, intelligence is the first and most important phase in the decision making process. With the escalation of information resources available to business executives, it is becoming imperative to explore the potential and challenges of using agent-based systems to support the intelligence phase of decision-making. This research examines UK executives’ perceptions of using agent-based support systems and the criteria for design and development of their “ideal” intelligent software agents. The study adopted an inductive approach using focus groups to generate a preliminary set of design criteria of “ideal” agents. It then followed a deductive approach using semi-structured interviews to validate and enhance the criteria. This qualitative research has generated unique insights into executives’ perceptions of the design and use of agent-based support systems. The systematic content analysis of qualitative data led to the proposal and validation of design criteria at three levels. The findings revealed the most desirable criteria for agent based support systems from the end users’ point view. The design criteria can be used not only to guide intelligent agent system design but also system evaluation

Autonomy in Video Games and Gamification

In the past decade, gamification (using game elements in non-gaming tasks to enhance motivation and engagement) has become a popular concept in many industries, but few studies have explored the principles under which it works. Self-determination theory suggests three psychological needs that gamification fulfills: competence, relatedness, and autonomy. Autonomy, a person\u27s perception that they have the ability to act however they choose, has emerged as an important, yet less-studied aspect in gamification. Inclusion of autonomy in gamification should foster engagement, enjoyment, and better performance. An experiment inspired by the above was carried out in which a sample of college students (N = 57) played a video game called Super Mario Bros. Crossover with either the choice to customize the aesthetics of their character and background (autonomy-supportive) or no choice of aesthetics (non-supportive). It was hypothesized that conditions involving more choice would lead to higher perceived autonomy and performance, and that perceived autonomy would be positively correlated with engagement, enjoyment, and performance. The manipulation resulted in no significant difference in perceived autonomy or performance, and perceived autonomy was only significantly positively correlated with enjoyment. Prior Super Mario Bros. experience was also found to positively correlate with perceived autonomy in the autonomy-supportive condition. The choice of aesthetics does not appear to have been sufficiently strong enough to increase perceived autonomy in this context

Design and Control of a Flight-Style AUV with Hovering Capability

The small flight-style Delphin AUV is designed to evaluate the performance of a long range survey AUV with the additional capability to hover and manoeuvre at slow speed. Delphin’s hull form is based on a scaled version of Autosub6000, and in addition to the main thruster and control surfaces at the rear of the vehicle, Delphin is equipped with four rim driven tunnel thrusters. In order to reduce the development cycle time, Delphin was designed to use commercial-off-the-shelf (COTS) sensors and thrusters interfaced to a standard PC motherboard running the control software within the MS Windows environment. To further simplify the development, the autonomy system uses the State-Flow Toolbox within the Matlab/Simulink environment. While the autonomy software is running, image processing routines are used for obstacle avoidance and target tracking, within the commercial Scorpion Vision software. This runs as a parallel thread and passes results to Matlab via the TCP/IP communication protocol. The COTS based development approach has proved effective. However, a powerful PC is required to effectively run Matlab and Simulink, and, due to the nature of the Windows environment, it is impossible to run the control in hard real-time. The autonomy system will be recoded to run under the Matlab Windows Real-Time Windows Target in the near future. Experimental results are used to demonstrating the performance and current capabilities of the vehicle are presented

Federated Embedded Systems – a review of the literature in related fields

This report is concerned with the vision of smart interconnected objects, a vision that has attracted much attention lately. In this paper, embedded, interconnected, open, and heterogeneous control systems are in focus, formally referred to as Federated Embedded Systems. To place FES into a context, a review of some related research directions is presented. This review includes such concepts as systems of systems, cyber-physical systems, ubiquitous computing, internet of things, and multi-agent systems. Interestingly, the reviewed fields seem to overlap with each other in an increasing number of ways