Toward a Dependability Case Language and Workflow for a Radiation Therapy System

We present a near-future research agenda for bringing a suite of modern programming-languages verification tools - specifically interactive theorem proving, solver-aided languages, and formally defined domain-specific languages - to the development of a specific safety-critical system, a radiotherapy medical device. We sketch how we believe recent programming-languages research advances can merge with existing best practices for safety-critical systems to increase system assurance and developer productivity. We motivate hypotheses central to our agenda: That we should start with a single specific system and that we need to integrate a variety of complementary verification and synthesis tools into system development

Practical Verification of Safety-Critical Systems

Thesis (Ph.D.)--University of Washington, 2018Software-based control systems operate scientific equipment worth millions of dollars and even safety-critical medical devices, making them good targets for strong formal verification techniques. However, these systems are rarely verified in practice. We identify three key challenges hindering the application of verification to real-world control systems and present solutions to each. First, safety properties of control systems often rely on correct operation and interaction of several heterogeneous hardware and software components. No single analysis tool can reason about all types of components. We present techniques, based on the established practice of safety case construction, for building a machine-checkable safety case that combines concrete evidence about the system implementation derived from multiple analysis tools. Using these techniques, we uncovered safety-critical flaws in a prerelease version of control software for the Clinical Neutron Therapy System (CNTS), a radiotherapy installation. Second, software components of control systems are often developed using proprietary or domain-specific languages for which no formal semantics yet exist. We present a methodology for rapidly developing language semantics, allowing application of formal verification techniques in languages that have received little previous study. We used this methodology to develop semantics for Python and for the EPICS dataflow language, suitable for analyzing components of the CNTS control software. Third, for control system software written in specialized languages, often no verified language implementations are available. We present a new technique for developing verified compilers that combines a verified denotation function with a verified extraction procedure to achieve high run-time performance with low verification effort. We demonstrate the effectiveness of this technique by developing a verified compiler for a fragment of the EPICS dataflow language and using it to compile portions of the CNTS control software

Collaborative verification of information flow for a high-assurance app store.

ABSTRACT Current app stores distribute some malware to unsuspecting users, even though the app approval process may be costly and timeconsuming. High-integrity app stores must provide stronger guarantees that their apps are not malicious. We propose a verification model for use in such app stores to guarantee that the apps are free of malicious information flows. In our model, the software vendor and the app store auditor collaborate -each does tasks that are easy for her/him, reducing overall verification cost. The software vendor provides a behavioral specification of information flow (at a finer granularity than used by current app stores) and source code annotated with information-flow type qualifiers. A flow-sensitive, context-sensitive information-flow type system checks the information flow type qualifiers in the source code and proves that only information flows in the specification can occur at run time. The app store auditor uses the vendor-provided source code to manually verify declassifications. We have implemented the information-flow type system for Android apps written in Java, and we evaluated both its effectiveness at detecting information-flow violations and its usability in practice. In an adversarial Red Team evaluation, we analyzed 72 apps (576,000 LOC) for malware. The 57 Trojans among these had been written specifically to defeat a malware analysis such as ours. Nonetheless, our information-flow type system was effective: it detected 96% of malware whose malicious behavior was related to information flow and 82% of all malware. In addition to the adversarial evaluation, we evaluated the practicality of using the collaborative model. The programmer annotation burden is low: 6 annotations per 100 LOC. Every sound analysis requires a human to review potential false alarms, and in our experiments, this took 30 minutes per 1,000 LOC for an auditor unfamiliar with the app