On the naturalness of software

Natural languages like English are rich, complex, and powerful. The highly creative and graceful use of languages like English and Tamil, by masters like Shakespeare and Avvaiyar, can certainly delight and inspire. But in practice, given cognitive constraints and the exigencies of daily life, most human utterances are far simpler and much more repetitive and predictable. In fact, these utterances can be very usefully modeled using modern statistical methods. This fact has led to the phenomenal success of statistical approaches to speech recognition, natural language translation, question-answering, and text mining and comprehension. We begin with the conjecture that most software is also natural, in the sense that it is created by humans at work, with all the attendant constraints and limitations---and thus, like natural language, it is also likely to be repetitive and predictable. We then proceed to ask whether (a) code can be usefully modeled by statistical language models and (b) such models can be leveraged to support software engineers. Using the widely adopted n-gram model, we provide empirical evidence supportive of a positive answer to both these questions. We show that code is also very regular, and, in fact, even more so than natural languages. As an example use of the model, we have developed a simple code completion engine for Java that, despite its simplicity, already improves Eclipse's completion capability. We conclude the paper by laying out a vision for future research in this area

Ontology Support for a Lightweight Formal Verification System

The usability of verification systems is becoming increasingly important, and the effective integration of ontologies of formal facts (definitions, propositions, and syntactic idioms) into machine verification systems will likely play a role in improving the usability of such systems. The "aartifact" lightweight verification system utilizes an ontology of formal propositions in order to support lightweight verification of formal arguments that involve common mathematical concepts. The ontology is stored within a relational database, and can be assembled and extended using a simple web interface by contributors who are domain experts. The database can be compiled into two separate components of the "aartifact" system: a verifier component that computes congruence closures of expressions containing relations and predicates found in the ontology, and a JavaScript application that interactively presents to users information about the constants, operators, relations, predicates, syntactic constructs, and idioms found in the ontology (and, thus, supported by the verifier). In this way, the database serves to improve both the verification system's capacity to infer implicit applications of logical propositions within a user's formal argument, and to inform users in a context-aware and structured manner of the verification system's capabilities and limitations

Seamless composition and integration: a perspective on formal methods research

Formal methods are now a central component of computer-science education and research. However, there will always be advances in mathematical logic -- a.k.a. `formal methods' among computer scientists -- leading to advances in reliable, safe and secure computing. There are many research directions that will promote the impact of formal methods on computer science in significant and novel ways. We outline two directions, each associated with its own research challenges, that are complementary to the current state-of-the-art: one of composability and one of integration, each considered in a specific context drawn from our own recent research and teaching experience. We try to clarify why the study and ultimate resolution of these two challenges hold the promise of important breakthroughs in the accessability of formal methods and, ultimately, their applicability.National Science Foundation (CCF-0820138

A User-friendly Interface for a Lightweight Verification System

User-friendly interfaces can play an important role in bringing the benefits of a machine-readable representation of formal arguments to a wider audience. The "aartifact" system is an easy-to-use lightweight verifier for formal arguments that involve logical and algebraic manipulations of common mathematical concepts. The system provides validation capabilities by utilizing a database of propositions governing common mathematical concepts. The "aartifact" system's multi-faceted interactive user interface combines several approaches to user-friendly interface design: (1) a familiar and natural syntax based on existing conventions in mathematical practice, (2) a real-time keyword-based lookup mechanism for interactive, context-sensitive discovery of the syntactic idioms and semantic concepts found in the system's database of propositions, and (3) immediate validation feedback in the form of reformatted raw input. The system's natural syntax and database of propositions allow it to meet a user's expectations in the formal reasoning scenarios for which it is intended. The real-time keyword-based lookup mechanism and validation feedback allow the system to teach the user about its capabilities and limitations in an immediate, interactive, and context-aware manner

A User-friendly Interface for a Lightweight Verification System

User-friendly interfaces can play an important role in bringing the benefits of a machine-readable representation of formal arguments to a wider audience. The "aartifact" system is an easy-to-use lightweight verifier for formal arguments that involve logical and algebraic manipulations of common mathematical concepts. The system provides validation capabilities by utilizing a database of propositions governing common mathematical concepts. The "aartifact" system's multi-faceted interactive user interface combines several approaches to user-friendly interface design: (1) a familiar and natural syntax based on existing conventions in mathematical practice, (2) a real-time keyword-based lookup mechanism for interactive, context-sensitive discovery of the syntactic idioms and semantic concepts found in the system's database of propositions, and (3) immediate validation feedback in the form of reformatted raw input. The system's natural syntax and database of propositions allow it to meet a user's expectations in the formal reasoning scenarios for which it is intended. The real-time keyword-based lookup mechanism and validation feedback allow the system to teach the user about its capabilities and limitations in an immediate, interactive, and context-aware manner

Fostering the Diversity of Exploratory Testing in Web Applications

International audienceExploratory testing (ET) is a software testing approach that complements automated testing by leveraging business expertise. It has gained momentum over the last decades as it appeals testers to exploit their business knowledge to stress the system under test (SUT). Exploratory tests, unlike automated tests, are defined and executed on-the-fly by testers.Testers who perform exploratory tests may be biased by their past experience and therefore may miss anomalies or unusual interactions proposed by the SUT. This is even more complex in the context of web applications, which typically expose a huge number of interaction paths to their users. As testers of these applications cannot remember all the sequences of interactions they performed, they may fail to deeply explore the application scope.This paper therefore introduces a new approach that assists testers in widely exploring any web application. In particular, our approach monitors the online interactions performed by the testers to suggest in real-time the probabilities of performing next interactions. Looking at these probabilities, we claim that the testers who favour interactions that have a low probability(because they were rarely performed), will increase the diversity of their explorations. Our approach defines a prediction model, based on n-grams, that encodes the history of past interactions and that supports the estimation of the probabilities. Integrated within a web browser extension, it automatically and transparently injects feedback within the application itself. We conduct a controlled experiment and a qualitative study to assess our approach. Results show that it prevents testers to be trapped in already tested loops, and succeeds to assist them in performing deeper explorations of the SUT