Extraction of System States from Natural Language Requirements

In recent years, simulations have proven to be an important means to verify the behavior of complex software systems. The different states of a system are monitored in the simulations and are compared against the requirements specification. So far, system states in natural language requirements cannot be automatically linked to signals from the simulation. However, the manual mapping between requirements and simulation is a time-consuming task. Named-entity Recognition is a sub-task from the field of automated information retrieval and is used to classify parts of natural language texts into categories. In this paper, we use a self-trained Named-entity Recognition model with Bidirectional LSTMs and CNNs to extract states from requirements specifications. We present an almost entirely automated approach and an iterative semi-automated approach to train our model. The automated and iterative approach are compared and discussed with respect to the usual manual extraction. We show that the manual extraction of states in 2,000 requirements takes nine hours. Our automated approach achieves an F1-score of 0.51 with 15 minutes of manual work and the iterative approach achieves an F1-score of 0.62 with 100 minutes of work

A comparative analysis of decision trees vis-a-vis other computational data mining techniques in automotive insurance fraud detection

The development and application of computational data mining techniques in financial fraud detection and business failure prediction has become a popular cross-disciplinary research area in recent times involving financial economists, forensic accountants and computational modellers. Some of the computational techniques popularly used in the context of - financial fraud detection and business failure prediction can also be effectively applied in the detection of fraudulent insurance claims and therefore, can be of immense practical value to the insurance industry. We provide a comparative analysis of prediction performance of a battery of data mining techniques using real-life automotive insurance fraud data. While the data we have used in our paper is US-based, the computational techniques we have tested can be adapted and generally applied to detect similar insurance frauds in other countries as well where an organized automotive insurance industry exists

Large-Scale Pattern-Based Information Extraction from the World Wide Web

Extracting information from text is the task of obtaining structured, machine-processable facts from information that is mentioned in an unstructured manner. It thus allows systems to automatically aggregate information for further analysis, efficient retrieval, automatic validation, or appropriate visualization. This work explores the potential of using textual patterns for Information Extraction from the World Wide Web

Transfer of Reinforcement Learning-Based Controllers from Model- to Hardware-in-the-Loop

The process of developing control functions for embedded systems is resource-, time-, and data-intensive, often resulting in sub-optimal cost and solutions approaches. Reinforcement Learning (RL) has great potential for autonomously training agents to perform complex control tasks with minimal human intervention. Due to costly data generation and safety constraints, however, its application is mostly limited to purely simulated domains. To use RL effectively in embedded system function development, the generated agents must be able to handle real-world applications. In this context, this work focuses on accelerating the training process of RL agents by combining Transfer Learning (TL) and X-in-the-Loop (XiL) simulation. For the use case of transient exhaust gas re-circulation control for an internal combustion engine, use of a computationally cheap Model-in-the-Loop (MiL) simulation is made to select a suitable algorithm, fine-tune hyperparameters, and finally train candidate agents for the transfer. These pre-trained RL agents are then fine-tuned in a Hardware-in-the-Loop (HiL) system via TL. The transfer revealed the need for adjusting the reward parameters when advancing to real hardware. Further, the comparison between a purely HiL-trained and a transferred agent showed a reduction of training time by a factor of 5.9. The results emphasize the necessity to train RL agents with real hardware, and demonstrate that the maturity of the transferred policies affects both training time and performance, highlighting the strong synergies between TL and XiL simulation

Enforcing Behavioral Profiles through Software-Defined Networks in the Industrial Internet of Things

The fourth industrial revolution is being mainly driven by the integration of Internet of Things (IoT) technologies to support the development lifecycle of systems and products. Despite the well-known advantages for the industry, an increasingly pervasive industrial ecosystem could make such devices an attractive target for potential attackers. Recently, the Manufacturer Usage Description (MUD) standard enables manufacturers to specify the intended use of their devices, thereby restricting the attack surface of a certain system. In this direction, we propose a mechanism to manage securely the obtaining and enforcement of MUD policies through the use of a Software-Defined Network (SDN) architecture. We analyze the applicability and advantages of the use of MUD in industrial environments based on our proposed solution, and provide an exhaustive performance evaluation of the required processes

Analysis and evaluation of embedded graphics solutions for critical systems

En el camp dels sistemes crítics, que inclou l'automotriu, l'aviònica i els sistemes espacials, es necessita més capacitat de computació per aportar tant valor funcional com seguretat addicional. Per aconseguir-ho, la indústria està considerant noves arquitectures per futurs sistemes crítics. Una de les possibles opcions és l'ús de targetes gràfiques mòbils, que tenen un rendiment excel·lent per tasques computacionals complexes i un baix nivell de consum. Per desgràcia, les eines actuals de desenvolupament per programació de propòsit general de targetes gràfiques com CUDA o OpenCL no compleixen amb les regulacions dels estàndards de seguretat dels sistemes crítics segurs. Per altra banda, hi ha altres solucions per programar per gràfics, com ara OpenGL SC 2 i Brook Auto, que són fàcils de certificar. En aquest projecte, analitzem aquestes solucions per programar per targetes gràfiques i explorem els diferents aspectes del desenvolupament de programari de propòsit general amb elles. Us presentem la nostra experiència adaptant codi de dues aplicacions de dos sectors diferents de sistemes crítics, l'aviònica i els sistemes espacials, a diferents \textit{APIs} (OpenGL 2, OpenGL ES 2, OpenGL SC 2 i Brook Auto) i l'avaluació de les versions que nosaltres hem generat. En funcionalitat i rendiment, no s'ha observat cap diferència, tot i que sí que hem notat un gran salt comparatiu en la complexitat del desenvolupament i la productivitat entre eines orientades només a sistemes gràfics i Brook Auto.In the safety-critical systems domain, which includes automotive, avionics and space systems, more compute power is needed to provide additional functional value and safety. In order to achieve this, new hardware architectures are considered from industry for future critical systems. One of this approaches is the use of mobile GPUs, which have excellent performance capabilities for intensive computational tasks and low-power consumption. However, current programming models for general purpose programming of GPUs like CUDA and OpenCL do not comply with the safety standards of safety critical systems. On the other hand, there are alternative programming solutions based on graphics, namely OpenGL SC 2 and Brook Auto, which are certification-friendly. In this thesis, we perform an analysis of these safety-critical programming models for GPUs and we explore the different aspects of the development of general purpose software in them. We present our experience with porting two applications from two distinct safety-critical domains, aerospace and avionics, in several graphics-based APIs (OpenGL 2, OpenGL ES 2, OpenGL SC 2 and Brook Auto) and the evaluation of our produced versions. In terms of functionality and performance, no difference has been observed, whereas we noticed a big gap in the development complexity and productivity between pure graphics solutions and Brook Auto

A Framework for the Verification and Validation of Artificial Intelligence Machine Learning Systems

An effective verification and validation (V&V) process framework for the white-box and black-box testing of artificial intelligence (AI) machine learning (ML) systems is not readily available. This research uses grounded theory to develop a framework that leads to the most effective and informative white-box and black-box methods for the V&V of AI ML systems. Verification of the system ensures that the system adheres to the requirements and specifications developed and given by the major stakeholders, while validation confirms that the system properly performs with representative users in the intended environment and does not perform in an unexpected manner. Beginning with definitions, descriptions, and examples of ML processes and systems, the research results identify a clear and general process to effectively test these systems. The developed framework ensures the most productive and accurate testing results. Formerly, and occasionally still, the system definition and requirements exist in scattered documents that make it difficult to integrate, trace, and test through V&V. Modern system engineers along with system developers and stakeholders collaborate to produce a full system model using model-based systems engineering (MBSE). MBSE employs a Unified Modeling Language (UML) or System Modeling Language (SysML) representation of the system and its requirements that readily passes from each stakeholder for system information and additional input. The comprehensive and detailed MBSE model allows for direct traceability to the system requirements. xxiv To thoroughly test a ML system, one performs either white-box or black-box testing or both. Black-box testing is a testing method in which the internal model structure, design, and implementation of the system under test is unknown to the test engineer. Testers and analysts are simply looking at performance of the system given input and output. White-box testing is a testing method in which the internal model structure, design, and implementation of the system under test is known to the test engineer. When possible, test engineers and analysts perform both black-box and white-box testing. However, sometimes testers lack authorization to access the internal structure of the system. The researcher captures this decision in the ML framework. No two ML systems are exactly alike and therefore, the testing of each system must be custom to some degree. Even though there is customization, an effective process exists. This research includes some specialized methods, based on grounded theory, to use in the testing of the internal structure and performance. Through the study and organization of proven methods, this research develops an effective ML V&V framework. Systems engineers and analysts are able to simply apply the framework for various white-box and black-box V&V testing circumstances