FLAVERS: a Finite State Verification Technique for Software Systems

Software systems are increasing in size and complexity and, subsequently, are becoming ever more difficult to validate. Finite State Verification (FSV) has been gaining credibility and attention as an alternative to testing and to formal verification approaches based on theorem proving. There has recently been a great deal of excitement about the potential for FSV approaches to prove properties about hardware descriptions but, for the most part, these approaches do not scale adequately to handle the complexity usually found in software. In this paper, we describe an FSV approach that creates a compact and conservative, but imprecise, model of the system being analyzed, and then assists the analyst in adding additional details as guided by previous analysis results. This paper describes this approach and a prototype implementation, called FLAVERS, presents a detailed example, and then provides some experimental results demonstrating scalability

Verifying Properties of Process Definitions

It seems important that the complex processes that synergize humans and computers to solve widening classes of societal problems be subjected to rigorous analysis. One approach is to use a process definition language to specify these processes and to then use analysis techniques to evaluate these definitions for important correctness properties. Because humans demand flexibility in their participation in complex processes, process definition languages must incorporate complicated control structures, such as various concurrency, choice, reactive control, and exception mechanisms. The underlying complexity of these control abstractions, however, often confounds the users’ intuitions as well as complicates any analysis. Thus, the control abstraction complexity in process definition languages presents analysis challenges beyond those posed by traditional programming languages. This paper explores some of the difficulties of analyzing process definitions. We explore issues arising when applying the FLAVERS finite state verification system to processes written in the Little-JIL process definition language and illustrate these issues using a realistic auction example. Although we employ a particular process definition language and analysis technique, our results seem more generally applicable

Automatic Derivation of Requirements for Components Used in Human-Intensive Systems

Human-intensive systems (HISs), where humans must coordinate with each other along with software and/or hardware components to achieve system missions, are increasingly prevalent in safety-critical domains (e.g., healthcare). Such systems are often complex, involving aspects such as concurrency and exceptional situations. For these systems, it is often difficult but important to determine requirements for the individual components that are necessary to ensure the system requirements are satisfied. In this thesis, we investigated an approach that employs interface synthesis methods developed for software systems to automatically derive such requirements for components used in HISs. In previous work, we investigated a requirement deriver that employs a regular language learning algorithm to iteratively refine the derived requirement based on counterexamples generated by model checking techniques. Since this learning-based requirement deriver often did not scale well, we investigated several learning and model checking optimizations. These optimizations significantly improved performance but affected the counterexample generation heuristics, often widely varying the permissiveness of the derived requirements. For comparison purposes, we investigated a direct requirement deriver that was purported to have poor performance but guarantees the derived requirements are adequately permissive, conceptually meaning the requirements are permissive as possible without violating the system requirements. For our evaluation, we applied these requirement derivers to case studies in two important domains, healthcare and election administration. Based on this evaluation, the direct requirement deriver with all optimizations applied had reasonable performance and ensures the derived requirements are adequately permissive. For the learning-based requirement deriver, many of the optimizations and heuristics have been presented previously, but we recommend how to selectively combine them to obtain reasonable performance while usually producing the adequately permissive derived requirements. Since such derived requirements often reflect the system complexity, these requirements can be easily misunderstood. Thus, we also investigated building views of the requirements that abstract away or highlight certain aspects to try to improve their understandability. Each single view appears to improve understandability and the multiple views seem to complement each other further improving understandability. Such derived requirements and their views can be used to safely develop and deploy the components used in HISs

Specification and Analysis of Resource Utilization Policies for Human-Intensive Systems

Contemporary systems often require the effective support of many types of resources, each governed by complex utilization policies. Sound management of these resources plays a key role in assuring that these systems achieve their key goals. To help system developers make sound resource management decisions, I provide a resource utilization policy specification and analysis framework for (1) specifying very diverse kinds of resources and their potentially complex resource utilization policies, (2) dynamically evaluating the policies’ effects on the outcomes achieved by systems utilizing the resources, and (3) formally verifying various kinds of properties of these systems. Resource utilization policies range from simple, e.g., first-in-first-out, to extremely complex, responding to changes in system environment, state, and stimuli. Further, policies may at times conflict with each other, requiring conflict resolution strategies that add extra complexity. Prior specification approaches rely on relatively simple resource models that prevent the specification of complex utilization and conflict resolution policies. My approach (1) separates resource utilization policy concerns from resource characteristic and request specifications, (2) creates an expressive specification notation for constraint policies, and (3) creates a resource constraint conflict resolution capability. My approach enables creating specifications of policies that are sufficiently precise and detailed to support static and dynamic analyses of how these policies affect the properties of systems constrained or governed by these policies. I provide a process- and resource-aware discrete-event simulator for simulating system executions that adhere to policies of resource utilization. The simulator integrates the existing JSim simulation engine with a separate resource management system. The separate architectural component makes it easy to keep track of resource utilization traces during a simulation run. My simulation framework facilitates considerable flexibility in the evaluation of diverse resource management decisions and powerful dynamic analyses. Dynamic verification through simulation is inherently limited because of the impossibility of exhaustive simulation of all scenarios. I complement this approach with static verification. Prior static resource analysis has supported the verification only of relatively simple resource utilization policies. My research utilizes powerful model checking techniques, building on the existing FLAVERS model checking tool, to verify properties of complex systems that are also verified to conform to complex resource utilization policies. My research demonstrates how to use systems such as FLAVERS to verify adherence to complex resource utilization policies as well as overall system properties, such as the absence of resource leak and resource deadlock. I evaluated my approach working with a hospital emergency department domain expert, using detailed, expert-developed models of the processes and resource utilization policies of an emergency department. In doing this, my research demonstrates how my framework can be effective in guiding the domain expert towards making sound decisions about policies for the management of hospital resources, while also providing rigorously-based assurances that the guidance is reliable and well-founded. My research makes the following contributions: (1) a specification language for resources and resource utilization policies for human-intensive systems, (2) a process- and resource-aware discrete-event simulation engine that creates simulations that adhere to the resource utilization policies, allowing for the dynamic evaluation of resource utilization policies, (3) a process- and resource-aware model checking technique that formally verifies system properties and adherence to resource utilization policies, and (4) validated and verified specifications of an emergency department healthcare system, demonstrating the utility of my approach

Proceedings of Monterey Workshop 2001 Engineering Automation for Sofware Intensive System Integration

The 2001 Monterey Workshop on Engineering Automation for Software Intensive System Integration was sponsored by the Office of Naval Research, Air Force Office of Scientific Research, Army Research Office and the Defense Advance Research Projects Agency. It is our pleasure to thank the workshop advisory and sponsors for their vision of a principled engineering solution for software and for their many-year tireless effort in supporting a series of workshops to bring everyone together.This workshop is the 8 in a series of International workshops. The workshop was held in Monterey Beach Hotel, Monterey, California during June 18-22, 2001. The general theme of the workshop has been to present and discuss research works that aims at increasing the practical impact of formal methods for software and systems engineering. The particular focus of this workshop was "Engineering Automation for Software Intensive System Integration". Previous workshops have been focused on issues including, "Real-time & Concurrent Systems", "Software Merging and Slicing", "Software Evolution", "Software Architecture", "Requirements Targeting Software" and "Modeling Software System Structures in a fastly moving scenario".Office of Naval ResearchAir Force Office of Scientific Research Army Research OfficeDefense Advanced Research Projects AgencyApproved for public release, distribution unlimite

A benchmark for evaluating software engineering techniques for improving medical processes

The software engineering and medical informatics communi-ties have been developing a range of approaches for reason-ing about medical processes. To facilitate the comparison of such approaches, it would be desirable to have a set of medical examples, or benchmarks, that are easily available, described in considerable detail, and characterized in terms of the real-world complexities they capture. This paper presents one such benchmark and discusses a list of desider-ata that medical benchmarks can be evaluated against

Specification and Analysis of Resource Utilization Policies for Human-Intensive Systems (Extended Abstract)

Societal processes, such as those used in healthcare, typically depend on the effective utilization of resources, both human and non-human. Sound policies for the management of these resources are crucial in assuring that these processes achieve their goals. But complex utilization policies may govern the use of such resources, increasing the difficulty of accurately incorporating resource considerations into complex processes. This dissertation presents an approach to the specification, allocation, and analysis of the management of such resources

Evaluating Software Architectures: Development Stability and Evolution

We survey seminal work on software architecture evaluationmethods. We then look at an emerging class of methodsthat explicates evaluating software architectures forstability and evolution. We define architectural stabilityand formulate the problem of evaluating software architecturesfor stability and evolution. We draw the attention onthe use of Architectures Description Languages (ADLs) forsupporting the evaluation of software architectures in generaland for architectural stability in specific

Variation in Human-Intensive Systems: a Conceptual Framework for Characterizing, Modeling, and Analyzing Families of Systems

A system model---namely a formal definition of the coordination of people, hardware devices, and software components performing activities, using resources and artifacts, and producing various outputs---can aid understanding of the real-world system it models. Complex real-world systems, however, exhibit considerable amounts of variation that can be difficult or impossible to represent within a single model. This dissertation evaluates the hypothesis that the careful characterization and representation of system variation can aid in the generation and analysis of concrete system instances related to one another in specified ways and manifesting different kinds of variation. When a set of closely related systems can be characterized by a compelling set-membership criterion, it is often useful and appropriate to characterize the set as a family of systems. In this dissertation, a variety of system variation requirements and corresponding needs for family specification criteria are identified. We focus on two specific kinds of variation, namely functional and agent variation, and suggest an approach for meeting these needs both at the level of requirements specification (problem-level variation), as well as at the level of implementation specification (solution-level variation). We present a framework for generating and analyzing new system instances, using the Little-JIL process definition language as an experimental vehicle to study what process definition language capabilities are necessary to support the explicit modeling of variation at the solution level, and thereby to address needs at the problem level. We define a formal notation for specifying functional and agent variation in human-intensive processes and describe a prototype system to accommodate this specification within an existing modeling framework. Once a family of systems is formally defined and characterized at the solution level, different analysis techniques can be applied to make assurances that all members of the family share certain kinds of properties. These analysis results can then be used to inform variation needs at the problem level. To evaluate the applicability of the approach, we study and model the variation observed in two real-world, human-intensive systems from the domains of conflict resolution and elections. Both case study domains have been observed to employ functional variants of their processes, and, given their complex coordination of human and software agents, both domains require agent variation, therefore fostering a fruitful application of our approach