Design for validation: An approach to systems validation

Every complex system built is validated in some manner. Computer validation begins with review of the system design. As systems became too complicated for one person to review, validation began to rely on the application of adhoc methods by many individuals. As the cost of the changes mounted and the expense of failure increased, more organized procedures became essential. Attempts at devising and carrying out those procedures showed that validation is indeed a difficult technical problem. The successful transformation of the validation process into a systematic series of formally sound, integrated steps is necessary if the liability inherent in the future digita-system-based avionic and space systems is to be minimized. A suggested framework and timetable for the transformtion are presented. Basic working definitions of two pivotal ideas (validation and system life-cyle) are provided and show how the two concepts interact. Many examples are given of past and present validation activities by NASA and others. A conceptual framework is presented for the validation process. Finally, important areas are listed for ongoing development of the validation process at NASA Langley Research Center

Towards a design-by-contract based approach for realizable connector-centric software architectures

Despite being a widely-used language for specifying software systems, UML remains less than ideal for software architectures. Architecture description languages (ADLs) were developed to provide more comprehensive support. However, so far the application of ADLs in practice has been impeded by at least one of the following problems: (i) advanced formal notations, (ii) lack of support for complex connectors, and (iii) potentially unrealizable designs. In this paper we propose a new ADL that is based on Design-by-Contract (DbC) for specifying software architectures. While DbC promotes a formal and precise way of specifying system behaviours, it is more familiar to practising developers, thus allowing for a more comfortable way of specifying architectures than using process algebras. Furthermore, by granting connectors a first-class status, our ADL allows designers to specify not only simple interaction mechanisms as connectors but also complex interaction protocols. Finally, in order to ensure that architectural designs are always realizable we eliminate potentially unrealizable constructs in connector specifications (the connector “glue”)

TriCheck: Memory Model Verification at the Trisection of Software, Hardware, and ISA

Memory consistency models (MCMs) which govern inter-module interactions in a shared memory system, are a significant, yet often under-appreciated, aspect of system design. MCMs are defined at the various layers of the hardware-software stack, requiring thoroughly verified specifications, compilers, and implementations at the interfaces between layers. Current verification techniques evaluate segments of the system stack in isolation, such as proving compiler mappings from a high-level language (HLL) to an ISA or proving validity of a microarchitectural implementation of an ISA. This paper makes a case for full-stack MCM verification and provides a toolflow, TriCheck, capable of verifying that the HLL, compiler, ISA, and implementation collectively uphold MCM requirements. The work showcases TriCheck's ability to evaluate a proposed ISA MCM in order to ensure that each layer and each mapping is correct and complete. Specifically, we apply TriCheck to the open source RISC-V ISA, seeking to verify accurate, efficient, and legal compilations from C11. We uncover under-specifications and potential inefficiencies in the current RISC-V ISA documentation and identify possible solutions for each. As an example, we find that a RISC-V-compliant microarchitecture allows 144 outcomes forbidden by C11 to be observed out of 1,701 litmus tests examined. Overall, this paper demonstrates the necessity of full-stack verification for detecting MCM-related bugs in the hardware-software stack.Comment: Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating System

Prototyping Formal System Models with Active Objects

We propose active object languages as a development tool for formal system models of distributed systems. Additionally to a formalization based on a term rewriting system, we use established Software Engineering concepts, including software product lines and object orientation that come with extensive tool support. We illustrate our modeling approach by prototyping a weak memory model. The resulting executable model is modular and has clear interfaces between communicating participants through object-oriented modeling. Relaxations of the basic memory model are expressed as self-contained variants of a software product line. As a modeling language we use the formal active object language ABS which comes with an extensive tool set. This permits rapid formalization of core ideas, early validity checks in terms of formal invariant proofs, and debugging support by executing test runs. Hence, our approach supports the prototyping of formal system models with early feedback.Comment: In Proceedings ICE 2018, arXiv:1810.0205

A formal verification framework and associated tools for enterprise modeling : application to UEML

The aim of this paper is to propose and apply a verification and validation approach to Enterprise Modeling that enables the user to improve the relevance and correctness, the suitability and coherence of a model by using properties specification and formal proof of properties

A Historical Perspective on Runtime Assertion Checking in Software Development

This report presents initial results in the area of software testing and analysis produced as part of the Software Engineering Impact Project. The report describes the historical development of runtime assertion checking, including a description of the origins of and significant features associated with assertion checking mechanisms, and initial findings about current industrial use. A future report will provide a more comprehensive assessment of development practice, for which we invite readers of this report to contribute information

Nemos: a framework for axiomatic and executable specifications of memory consistency models

technical reportConforming to the underlying memory consistency rules is a fundamental require- ment for implementing shared memory systems and writing multiprocessor programs. In order to promote understanding and enable automated verification, it is highly desir- able that a memory model specification be both declarative and executable. We have developed a specification framework called Nemos (Non-operational yet Executable Memory Ordering Specifications), which employs a uniform notation based on predi- cate logic to define shared memory semantics in an axiomatic as well as compositional style. In this paper, we present this framework and discuss how constraint logic pro- gramming and SAT solving can be used to make these axiomatic specifications exe- cutable for memory model analysis, thus supporting precise specification and automatic execution in the same framework. To illustrate our approach, this paper formalizes a collection of well known memory models, including sequential consistency, coherence, PRAM, causal consistency, and processor consistency

A Taxonomy of Metamodel Hierarchies

In the context of software engineering and model-driven development in particular, metamodeling gains more and more importance. So far, no classifying study of theoretical metamodeling concepts and hierarchy design options has been conducted in order to establish a comprehensive set of interrelated design variables, i.e. a coherent design space. A well-designed metamodeling hierarchy is essential to avoid problems not easily noticeable, like ambiguous classification and the replication of concepts. This study aims at exploring the theoretical foundation and providing a taxonomy or a design space for constructing tailor-made metamodel hierarchies for specific problems areas and domains

Applying Formal Methods to Networking: Theory, Techniques and Applications

Despite its great importance, modern network infrastructure is remarkable for the lack of rigor in its engineering. The Internet which began as a research experiment was never designed to handle the users and applications it hosts today. The lack of formalization of the Internet architecture meant limited abstractions and modularity, especially for the control and management planes, thus requiring for every new need a new protocol built from scratch. This led to an unwieldy ossified Internet architecture resistant to any attempts at formal verification, and an Internet culture where expediency and pragmatism are favored over formal correctness. Fortunately, recent work in the space of clean slate Internet design---especially, the software defined networking (SDN) paradigm---offers the Internet community another chance to develop the right kind of architecture and abstractions. This has also led to a great resurgence in interest of applying formal methods to specification, verification, and synthesis of networking protocols and applications. In this paper, we present a self-contained tutorial of the formidable amount of work that has been done in formal methods, and present a survey of its applications to networking.Comment: 30 pages, submitted to IEEE Communications Surveys and Tutorial