Practical Verification of Safety-Critical Systems

Abstract

Software-based control systems operate scientific equipment worth millions of dollars and even safety-critical medical devices, making them good targets for strong formal verification techniques. However, these systems are rarely verified in practice. We identify three key challenges hindering the application of verification to real-world control systems and present solutions to each. First, safety properties of control systems often rely on correct operation and interaction of several heterogeneous hardware and software components. No single analysis tool can reason about all types of components. We present techniques, based on the established practice of safety case construction, for building a machine-checkable safety case that combines concrete evidence about the system implementation derived from multiple analysis tools. Using these techniques, we uncovered safety-critical flaws in a prerelease version of control software for the Clinical Neutron Therapy System (CNTS), a radiotherapy installation. Second, software components of control systems are often developed using proprietary or domain-specific languages for which no formal semantics yet exist. We present a methodology for rapidly developing language semantics, allowing application of formal verification techniques in languages that have received little previous study. We used this methodology to develop semantics for Python and for the EPICS dataflow language, suitable for analyzing components of the CNTS control software. Third, for control system software written in specialized languages, often no verified language implementations are available. We present a new technique for developing verified compilers that combines a verified denotation function with a verified extraction procedure to achieve high run-time performance with low verification effort. We demonstrate the effectiveness of this technique by developing a verified compiler for a fragment of the EPICS dataflow language and using it to compile portions of the CNTS control software.