Scientists in the MIST: Simplifying Interface Design for End Users

We are building a Malleable Interactive Software Toolkit (MIST), a tool set and infrastructure to simplify the design and construction of dynamically-reconfigurable (malleable) interactive software. Malleable software offers the end-user powerful tools to reshape their interactive environment on the fly. We aim to make the construction of such software straightforward, and to make reconfiguration of the resulting systems approachable and manageable to an educated, but non-specialist, user. To do so, we draw on a diverse body of existing research on alternative approaches to user interface (UI) and interactive software construction, including declarative UI languages, constraint-based programming and UI management, reflection and data-driven programming, and visual programming techniques

Emerging from the MIST: A Connector Tool for Supporting Programming by Non-programmers

Software development is an iterative process. As user re-quirements emerge software applications must be extended to support the new requirements. Typically, a programmer will add new code to an existing code base of an application to provide a new functionality. Previous research has shown that such extensions are easier when application logic is clearly separated from the user interface logic. Assuming that a programmer is already familiar with the existing code base, the task of writing the new code can be considered to be split into two sub-tasks: writing code for the application logic; that is, the actual functionality of the application; and writing code for the user interface that will expose the functionality to the end user. The goal of this research is to reduce the effort required to create a user interface once the application logic has been created, toward supporting scientists with minimal pro-gramming knowledge to be able to create and modify pro-grams. Using a Model View Controller based architecture, various model components which contain the application logic can be built and extended. The process of creating and extending the views (user interfaces) on these model components is simplified through the use of our Malleable Interactive Software Toolkit (MIST), a tool set an infrastructure intended to simplify the design and extension of dynamically reconfigurable interfaces. This paper focuses on one tool in the MIST suite, a connec-tor tool that enables the programmer to evolve the user interface as the application logic evolves by connecting related pieces of code together; either through simple drag-and-drop interactions or through the authoring of Python code. The connector tool exemplifies the types of tools in the MIST suite, which we expect will encourage collabora-tive development of applications by allowing users to inte-grate various components and minimizing the cost of de-veloping new user interfaces for the combined compo-nents

Natural Notation for the Domestic Internet of Things

This study explores the use of natural language to give instructions that might be interpreted by Internet of Things (IoT) devices in a domestic `smart home' environment. We start from the proposition that reminders can be considered as a type of end-user programming, in which the executed actions might be performed either by an automated agent or by the author of the reminder. We conducted an experiment in which people wrote sticky notes specifying future actions in their home. In different conditions, these notes were addressed to themselves, to others, or to a computer agent.We analyse the linguistic features and strategies that are used to achieve these tasks, including the use of graphical resources as an informal visual language. The findings provide a basis for design guidance related to end-user development for the Internet of Things.Comment: Proceedings of the 5th International symposium on End-User Development (IS-EUD), Madrid, Spain, May, 201

Knowledge Rich Natural Language Queries over Structured Biological Databases

Increasingly, keyword, natural language and NoSQL queries are being used for information retrieval from traditional as well as non-traditional databases such as web, document, image, GIS, legal, and health databases. While their popularity are undeniable for obvious reasons, their engineering is far from simple. In most part, semantics and intent preserving mapping of a well understood natural language query expressed over a structured database schema to a structured query language is still a difficult task, and research to tame the complexity is intense. In this paper, we propose a multi-level knowledge-based middleware to facilitate such mappings that separate the conceptual level from the physical level. We augment these multi-level abstractions with a concept reasoner and a query strategy engine to dynamically link arbitrary natural language querying to well defined structured queries. We demonstrate the feasibility of our approach by presenting a Datalog based prototype system, called BioSmart, that can compute responses to arbitrary natural language queries over arbitrary databases once a syntactic classification of the natural language query is made