Reducing the Usability Barrier to Specification and Verification

Abstract

A program specification is a contract between a client and a program, which describes how the program can or will behave given input from the client. In practice, formal (machine-verified) specifications take more effort and skill to write than informal natural language specifications. While some of the effort is essential to the task, much of the effort can be attributed design trade-offs and deficiencies in today's tools. This dissertation makes three research contributions. First, it identifies tool transparency and interoperability as primary barriers limiting the use of formal specification tools; it presents empirical, observational, and controlled studies characterizing the effects of each, and provides actionable recommendations for tool designers. Second, it introduces novel verification interface features to reduce the skill barrier to verification. A study with (relatively) low-skilled freelancers found that developers writing verified specifications with an interface that incorporates the features were more productive than those using a traditional interface. Third, to address the problems of tool transparency and interoperability in the more general context of the development tools, this dissertation presents a pipeline-based approach to end-user information discovery and analysis in the IDE. In a study with undergraduate students, the students were able to quickly learn to use a tool based on the approach to answer questions that arise during development.