Application of flight systems methodologies to the validation of knowledge-based systems

Flight and mission-critical systems are verified, qualified for flight, and validated using well-known and well-established techniques. These techniques define the validation methodology used for such systems. In order to verify, qualify, and validate knowledge-based systems (KBS's), the methodology used for conventional systems must be addressed, and the applicability and limitations of that methodology to KBS's must be identified. The author presents an outline of how this approach to the validation of KBS's is being developed and used at the Dryden Flight Research Facility of the NASA Ames Research Center

A NASA/RAE cooperation in the development of a real-time knowledge-based autopilot

As part of a US/UK cooperative aeronautical research program, a joint activity between the NASA Dryden Flight Research Facility and the Royal Aerospace Establishment on knowledge-based systems was established. This joint activity is concerned with tools and techniques for the implementation and validation of real-time knowledge-based systems. The proposed next stage of this research is described, in which some of the problems of implementing and validating a knowledge-based autopilot for a generic high-performance aircraft are investigated

Physical-depth architectural requirements for generating universal photonic cluster states

Most leading proposals for linear-optical quantum computing (LOQC) use cluster states, which act as a universal resource for measurement-based (one-way) quantum computation (MBQC). In ballistic approaches to LOQC, cluster states are generated passively from small entangled resource states using so-called fusion operations. Results from percolation theory have previously been used to argue that universal cluster states can be generated in the ballistic approach using schemes which exceed the critical threshold for percolation, but these results consider cluster states with unbounded size. Here we consider how successful percolation can be maintained using a physical architecture with fixed physical depth, assuming that the cluster state is continuously generated and measured, and therefore that only a finite portion of it is visible at any one point in time. We show that universal LOQC can be implemented using a constant-size device with modest physical depth, and that percolation can be exploited using simple pathfinding strategies without the need for high-complexity algorithms.Comment: 18 pages, 10 figure

TABSAOND: A technique for developing agent-based simulation apps and online tools with nondeterministic decisions

Agent-based simulators (ABSs) have successfully allowed practitioners to estimate the outcomes of certain input circumstances in several domains. Although some techniques and processes provide hints about the construction of these systems, some aspects have not been discussed yet in the literature. In this context, the current approach presents a technique for developing ABSs. Its focus is to guide practitioners in designing and implementing the decision-making processes of agents in nondeterministic scenarios. As an additional technological innovation, the ABSs are deployed as both mobile apps and online tools. This work illustrates the current approach with two case studies in the fields of (a) health and welfare and (b) tourism. These case studies have also been developed with the most similar technique from the literature for comparing both techniques. The presented technique improved the simulated outcomes in terms of their similarity with the real ones. The obtained ABSs were more efficient and reliable for large amounts of agents (e.g. 10,000 – 400,000 agents). The development time was lower. Both the framework and the implementation of a case study are freely distributed as open-source to facilitate the reproducibility of the experiments and to assist practitioners in applying the current approach

Engineering with logic: Rigorous test-oracle specification and validation for TCP/IP and the Sockets API

Conventional computer engineering relies on test-and-debug development processes, with the behavior of common interfaces described (at best) with prose specification documents. But prose specifications cannot be used in test-and-debug development in any automated way, and prose is a poor medium for expressing complex (and loose) specifications. The TCP/IP protocols and Sockets API are a good example of this: they play a vital role in modern communication and computation, and interoperability between implementations is essential. But what exactly they are is surprisingly obscure: their original development focused on “rough consensus and running code,” augmented by prose RFC specifications that do not precisely define what it means for an implementation to be correct. Ultimately, the actual standard is the de facto one of the common implementations, including, for example, the 15 000 to 20 000 lines of the BSD implementation—optimized and multithreaded C code, time dependent, with asynchronous event handlers, intertwined with the operating system, and security critical. This article reports on work done in the Netsem project to develop lightweight mathematically rigorous techniques that can be applied to such systems: to specify their behavior precisely (but loosely enough to permit the required implementation variation) and to test whether these specifications and the implementations correspond with specifications that are executable as test oracles . We developed post hoc specifications of TCP, UDP, and the Sockets API, both of the service that they provide to applications (in terms of TCP bidirectional stream connections) and of the internal operation of the protocol (in terms of TCP segments and UDP datagrams), together with a testable abstraction function relating the two. These specifications are rigorous, detailed, readable, with broad coverage, and rather accurate. Working within a general-purpose proof assistant (HOL4), we developed language idioms (within higher-order logic) in which to write the specifications: operational semantics with nondeterminism, time, system calls, monadic relational programming, and so forth. We followed an experimental semantics approach, validating the specifications against several thousand traces captured from three implementations (FreeBSD, Linux, and WinXP). Many differences between these were identified, as were a number of bugs. Validation was done using a special-purpose symbolic model checker programmed above HOL4. Having demonstrated that our logic-based engineering techniques suffice for handling real-world protocols, we argue that similar techniques could be applied to future critical software infrastructure at design time, leading to cleaner designs and (via specification-based testing) more robust and predictable implementations. In cases where specification looseness can be controlled, this should be possible with lightweight techniques, without the need for a general-purpose proof assistant, at relatively little cost.EPSRC Programme Grant EP/K008528/1 REMS: Rigorous Engineering for Mainstream Systems EPSRC Leadership Fellowship EP/H005633 (Sewell) Royal Society University Research Fellowship (Sewell) St Catharine's College Heller Research Fellowship (Wansbrough), EPSRC grant GR/N24872 Wide-area programming: Language, Semantics and Infrastructure Design EPSRC grant EP/C510712 NETSEM: Rigorous Semantics for Real Systems EC FET-GC project IST-2001-33234 PEPITO Peer-to-Peer Computing: Implementation and Theory CMI UROP internship support (Smith) EC Thematic Network IST-2001-38957 APPSEM 2 NICTA was funded by the Australian Government's Backing Australia's Ability initiative, in part through the Australian Research Council

Practical Model Checking of a Home Area Network System: Case Study

The integrated communication infrastructure is the core of the Smart Grid architecture. Its two-way communication and information flow provides this network with all needed resources in order to control and manage all connected components from the utility to the customer side. This latter, named the Home Area Network or HAN, is a dedicated network connecting smart devices inside the customer home, and using different solutions. In order to avoid problems and anomalies along the process life cycle of developing a new solution for HAN network, the modeling and validation is one of the most powerful tools to achieve this goal. This paper presents a practical case study of such validation. It intends to validate a HAN SDL model, described in a previous work, using model checking techniques. It introduces a method to translate the SDL model to a Promela model using an intermediate format IF. After the generation of the Promela model, verification is performed to ensure that some functional properties are satisfied. The desired properties are defined in Linear Temporal Logic (LTL), and DTSPIN (an extension of SPIN with discrete time) model checker is used to verify the correctness of the model

Virtual environment model generation for CPS goal verification using imitation learning

Cyber-Physical Systems (CPS) continuously interact with their physical environments through embedded software controllers that observe the environments and determine actions. Field Operational Tests (FOT) are essential to verify to what extent the CPS under analysis can achieve certain CPS goals, such as satisfying the safety and performance requirements, while interacting with the real operational environment. However, performing many FOTs to obtain statistically significant verification results is challenging due to its high cost and risk in practice. Simulation-based verification can be an alternative to address the challenge, but it still requires an accurate virtual environment model that can replace the real environment interacting with the CPS in a closed loop. In this paper, we propose ENVI (ENVironment Imitation), a novel approach to automatically generate an accurate virtual environment model, enabling efficient and accurate simulation-based CPS goal verification in practice.To do this, we first formally define the problem of the virtual environment model generation and solve it by leveraging Imitation Learning (IL), which has been actively studied in machine learning to learn complex behaviors from expert demonstrations. The key idea behind the model generation is to leverage IL for training a model that imitates the interactions between the CPS controller and its real environment as recorded in (possibly very small) FOT logs. We then statistically verify the goal achievement of the CPS by simulating it with the generated model. We empirically evaluate ENVI by applying it to the verification of two popular autonomous driving assistant systems. The results show that ENVI can reduce the cost of CPS goal verification while maintaining its accuracy by generating accurate environment models from only a few FOT logs. The use of IL in virtual environment model generation opens new research directions, further discussed at the end of the paper