Analysis of WIMP and Post WIMP Interactive Systems based on Formal Specification

While designing interactive software, the use of a formal specification technique is of great help by providing non-ambiguous, complete and concise descriptions. The advantages of using such a formalism is widened if it is provided by formal analysis techniques that allow to prove properties about the design, thus giving an early verification to the designer before the application is actually implemented. This paper presents how models built using the Interactive Cooperative Objects formalism (ICOs) are amenable to formal verification. The emphasis is on the behavioral part of the description of the interactive systems and more precisely on the properties at the interaction technique level. However, the process and the associated tools can be generalized to the other parts of the interactive systems (including the non-interactive parts)

Approches outillées pour le développement de systèmes interactifs intégrant les aspects sûreté de fonctionnement et utilisabilité

Since the Airbus A380 and with the introduction of ARINC 661 standard, the glass cockpits are being replaced by interactive cockpits, by allowing the crew to control aircraft systems through display unit by using keyboard and cursor control unit (KCCU). Currently only secondary aircraft systems which are non-critical are managed using such interactive cockpits. To be able to generalize such features to critical aircraft system, the main question remains to understand how to match dependability requirements for such systems while preserving usability properties. To reach the goal of using such interactive techniques within safety critical aircraft systems, our research work has followed three main directions. The first approach is to tend to zero default design, by realizing the precise and unambiguous description of software components of interactive system, using formal description technique. The second approach consists in the use of fault tolerant mechanisms, to treat design residual fault, physical fault or environmental fault. These fault tolerant mechanisms enable the continuity of service despite the occurrence of fault. The third approach is the clarification of the impact of different fault tolerant mechanisms on the usability of the interactive system. This clarification is done by using and analyzing task models, describing the user activity of the systemDepuis l'A380 et avec l'introduction du standard ARINC 661, les systèmes d'affichage et de contrôle des cockpits sont passés d'un rôle de simple afficheur, à celui d'un système interactif permettant à l'équipage d'interagir sur les écrans grâce à l'utilisation d'un ensemble clavier/dispositif de pointage appelé KCCU. L'utilisation de cette nouvelle capacité d'interaction est à ce jour limitée à des interactions avec des systèmes avions non critiques. Pour envisager son extension à des systèmes critiques il faut se poser la question du respect d'exigences de sureté de fonctionnement imposées à de tels systèmes sans pour autant diminuer son niveau d'utilisabilité. Dans cette optique, nous proposons dans le cadre de nos travaux de recherche, différentes approches pour contribuer au développement d'un tel système interactif critique. La première approche est de tendre vers une conception zéro défaut, en réalisant une description précise et non ambigüe des composants logiciels du système interactif en utilisant une technique de description formelle. La seconde approche est l'utilisation de techniques de tolérance aux fautes car il existe toujours des fautes résiduelles de conception, des fautes matérielles ou venant de l'environnement. Dans ce cas, l'utilisation de technique de tolérance aux fautes permet au système de continuer à remplir ses fonctions en dépit de l'occurrence de fautes. La troisième approche est l'explicitation de l'impact des différentes approches de tolérance aux fautes sur l'utilisabilité du système interactif. Cette explicitation est faite au travers de la réalisation et de l'analyse des modèles de tâche, décrivant l'activité de l'utilisateur du système

Approches outillées pour le développement des systèmes interactifs intégrant les aspects sûreté de fonctionnement et utilisabilité

Depuis l'A380 et avec l'introduction du standard ARINC 661, les systèmes d'affichage et de contrôle des cockpits sont passés d'un rôle de simple afficheur, à celui d'un système interactif permettant à l'équipage d'interagir sur les écrans grâce à l'utilisation d'un ensemble clavier/dispositif de pointage appelé KCCU. L'utilisation de cette nouvelle capacité d'interaction est à ce jour limitée à des interactions avec des systèmes avions non critiques. Pour envisager son extension à des systèmes critiques il faut se poser la question du respect d'exigences de sureté de fonctionnement imposées à de tels systèmes sans pour autant diminuer son niveau d'utilisabilité. Dans cette optique, nous proposons dans le cadre de nos travaux de recherche, différentes approches pour contribuer au développement d'un tel système interactif critique. La première approche est de tendre vers une conception zéro défaut, en réalisant une description précise et non ambigüe des composants logiciels du système interactif en utilisant une technique de description formelle. La seconde approche est l'utilisation de techniques de tolérance aux fautes car il existe toujours des fautes résiduelles de conception, des fautes matérielles ou venant de l'environnement. Dans ce cas, l'utilisation de technique de tolérance aux fautes permet au système de continuer à remplir ses fonctions en dépit de l'occurrence de fautes. La troisième approche est l'explicitation de l'impact des différentes approches de tolérance aux fautes sur l'utilisabilité du système interactif. Cette explicitation est faite au travers de la réalisation et de l'analyse des modèles de tâche, décrivant l'activité de l'utilisateur du système.Since the Airbus A380 and with the introduction of ARINC 661 standard, the glass cockpits are being replaced by interactive cockpits, by allowing the crew to control aircraft systems through display unit by using keyboard and cursor control unit (KCCU). Currently only secondary aircraft systems which are non-critical are managed using such interactive cockpits. To be able to generalize such features to critical aircraft system, the main question remains to understand how to match dependability requirements for such systems while preserving usability properties. To reach the goal of using such interactive techniques within safety critical aircraft systems, our research work has followed three main directions. The first approach is to tend to zero default design, by realizing the precise and unambiguous description of software components of interactive system, using formal description technique. The second approach consists in the use of fault tolerant mechanisms, to treat design residual fault, physical fault or environmental fault. These fault tolerant mechanisms enable the continuity of service despite the occurrence of fault. The third approach is the clarification of the impact of different fault tolerant mechanisms on the usability of the interactive system. This clarification is done by using and analyzing task models, describing the user activity of the system

A multi-formalism approach for model-based dynamic distribution of user interfaces of critical interactive systems.

International audienceEvolution in the context of use requires evolutions in the user interfaces even when they are currently used by operators. User Centered Development promotes reactive answers to this kind of evolutions either by software evolutions through iterative development approaches or at runtime by providing additional information to the operators such as contextual help for instance. This paper proposes a model-based approach to support proactive management of context of use evolutions. By proactive management we mean mechanisms in place to plan and implement evolutions and adaptations of the entire user interface (including behaviour) in a generic way. The approach proposed handles both concentration and distribution of user interfaces requiring both fusion of information into a single UI or fission of information into several ones. This generic model-based approach is exemplified on a safety critical system from space domain. It presents how the new user interfaces can be generated at runtime to provide a new user interface gathering in a single place all the information required to perform the task. These user interfaces have to be generated at runtime as new procedures (i.e. sequences of operations to be executed in a semi-autonomous way) can be defined by operators at any time in order to react to adverse events and to keep the space system in operation. Such contextual, activity-related user interfaces complement the original user interfaces designed for operating the command and control system. The resulting user interface thus corresponds to a distribution of user interfaces in a focus+context way improving usability by increasing both efficiency and effectiveness

Beyond Formal Methods for Critical Interactive Systems: Dealing with Faults at Runtime

International audienceFormal methods provide support for validation and verification of interactive systems by means of complete and unambiguous description of the envisioned system. They can also be used (for instance in the requirements/needs identification phase) to define precisely what the system should do and how it should meet user needs. If the entire development process in supported by formal methods (for instance as required by DO 178C [7] and its supplement 333 [8]) then classical formal method engineers would argue that the resulting software is defect free. However, events that are beyond the envelope of the specification may occur and trigger unexpected behaviors from the formally specified system resulting in failures. Sources of such failures can be permanent or transient hardware failures, due to (when such systems are deployed in the high atmosphere e.g. aircrafts or spacecrafts) natural faults triggered by alpha-particles from radioactive contaminants in the chips or neutron from cosmic radiation. This position paper proposes a complementary view to formal approaches first by presenting an overview of causes of unexpected events on the system side as well as on the human side and then by discussing approaches that could provide support for taking into account system faults and human errors at design time

Engineering Annotations: A Generic Framework For Gluing Design Artefacts in Models of Interactive Systems

International audienceAlong the design process of interactive system many intermediate artefacts (such as user interface prototypes, task models describing user work and activities, dialog models specifying system behavior, interaction models describing user interactions …) are created, tested, revised and improved until the development team produces a validated version of the full-fledged system. Indeed, to build interactive systems there is a need to use multiple artefacts/models (as they provide a complementary view). However, relevant information for describing the design solution and/or supporting design decisions (such as rational about the design, decisions made, recommendations, etc.) is not explicitly capturable in the models/artefacts, hence the need for annotations. Multi-artefacts approaches usually argue that a given information should only be present in one artefact to avoid duplication and increase maintainability of the artefacts. Nonetheless, annotations created on one artefact are usually relevant to other artefacts/models. So that, there is a need for tools and techniques to coordinate annotations across artefacts/models which is the contribution of the present work. In this paper, we propose a model-based approach that was conceived to handle annotations in a systematic way along the development process of interactive systems. As part of the solution, we propose an annotation model built upon the W3C's Web Annotation Data Model. The feasibility of the approach is demonstrated by means of a tool suite featuring a plugin, which has been deployed and tested over the multi-artefacts. The overall approach is illustrated on the design of an interactive cockpit application performing two design iterations. The contribution brings two main benefits for interactive systems engineering: i) it presents a generic pattern for integrating information in multiple usually heterogenous artefacts throughout the design process of interactive systems; and ii) it highlights the need for tools helping to rationalize and to document the various artefacts and the related decisions made during interactive systems design. CCS CONCEPTS • Human-centered computing • Human computer interaction (HCI

Définition d'un langage et d'une méthode pour la description et la spécification d'IHM post-W.I.M.P. pour les cockpits interactifs

Avec l'apparition de nouvelles technologies comme l'iPad, etc., nous rencontrons dans les logiciels grand public des interfaces de plus en plus riches et innovantes. Ces innovations portent à la fois sur la gestion des entrées (e. g. écrans multi-touch) et sur la gestion des sorties (e.g. affichage). Ces interfaces sont catégorisées de type post-WIMP et permettent d'accroitre la bande passante entre l'utilisateur et le système qu'il manipule. Plus précisément elles permettent à l'utilisateur de fournir plus rapidement des commandes au système et au système de présenter plus d'informations à l'utilisateur lui permettant par là-même de superviser des systèmes de complexité accrue. L'adoption par le grand public et le niveau de maturité de ces technos permet d'envisager leur intégration dans les systèmes critiques (comme les cockpits ou de façon plus générale les systèmes de commande et contrôle). Toutefois les aspects logiciels liés à ces technologies sont loin d'être maîtrisés comme le démontrent les nombreux dysfonctionnements rencontrés par leurs utilisateurs. Alors que ces derniers peuvent être tolérés pour des applications de jeux ou de divertissement elles ne sont pas acceptables dans le domaine des systèmes critiques présentés précédemment. La problématique de cette thèse porte précisément sur le développement de méthodes, langages, techniques et outils pour la conception et le développement de systèmes interactifs innovants et fiables. La contribution de cette thèse porte sur l'extension d'une notation formelle : ICO (Objets Coopératifs Interactifs) pour décrire de manières exhaustive et non ambiguë les techniques d'interactions multi-touch et la démonstrabilité de son application dans le cadre des applications multi-touch civils. Nous proposons en plus de cette notation, une méthode pour la conception et la validation de systèmes interactifs offrants des interactions multi-touch à leurs utilisateurs. Le fonctionnement de ces systèmes interactifs est basé sur une architecture générique permettant une structuration des modèles allant de la partie matérielle des périphériques d'entrées jusqu' à la partie applicative pour la commande et le contrôle de ces systèmes. Cet ensemble de contribution est appliqué sur un ensemble d'étude de ca dont la plus significative est une application de gestion météo pour un avion civil.With the advent of new technologies such as the iPad, general public software feature richer and more innovative interfaces. These innovations are both on the input layer (e.g. multi-touch screens) and on the output layer (e.g. display). These interfaces are categorized as post-W.I.M.P. type and allow to increase the bandwidth between the user and the system he manipulates. Specifically it allows the user to more quickly deliver commands to the system and the system to present more information to the user enabling him managing increasingly complex systems. The large use in the general public and the level of maturity of these technologies allows to consider their integration in critical systems (such as cockpits or more generally control and command systems). However, the software issues related to these technologies are far from being resolved judging by the many problems encountered by users. While the latter may be tolerated for gaming applications and entertainment, it is not acceptable in the field of critical systems described above. The problem of this thesis focuses specifically on the development of methods, languages, techniques and tools for the design and development of innovative and reliable interactive systems. The contribution of this thesis is the extension of a formal notation: ICO (Interactive Cooperative Object) to describe in a complete and unambiguous way multi-touch interaction techniques and is applied in the context of multi-touch applications for civilians aircrafts. We provide in addition to this notation, a method for the design and validation of interactive systems featuring multi-touch interactions. The mechanisms of these interactive systems are based on a generic architecture structuring models from the hardware part of the input devices up to the application part for the control and monitoring of these systems. This set of contribution is applied on a set of case studies, the most significant being an application for weather management in civilian aircrafts

Beyond Formal Methods for Critical Interactive Systems: Dealing with Faults at Runtime

Formal methods provide support for validation and verification of interactive systems by means of complete and unambiguous description of the envisioned system. They can also be used (for instance in the requirements/needs identification phase) to define precisely what the system should do and how it should meet user needs. If the entire development process in supported by formal methods (for instance as required by DO 178C [7] and its supplement 333 [8]) then classical formal method engineers would argue that the resulting software is defect free. However, events that are beyond the envelope of the specification may occur and trigger unexpected behaviors from the formally specified system resulting in failures. Sources of such failures can be permanent or transient hardware failures, due to (when such systems are deployed in the high atmosphere e.g. aircrafts or spacecrafts) natural faults triggered by alpha-particles from radioactive contaminants in the chips or neutron from cosmic radiation. This position paper proposes a complementary view to formal approaches first by presenting an overview of causes of unexpected events on the system side as well as on the human side and then by discussing approaches that could provide support for taking into account system faults and human errors at design time

The cockpit for the 21st century

Interactive surfaces are a growing trend in many domains. As one possible manifestation of Mark Weiser’s vision of ubiquitous and disappearing computers in everywhere objects, we see touchsensitive screens in many kinds of devices, such as smartphones, tablet computers and interactive tabletops. More advanced concepts of these have been an active research topic for many years. This has also influenced automotive cockpit development: concept cars and recent market releases show integrated touchscreens, growing in size. To meet the increasing information and interaction needs, interactive surfaces offer context-dependent functionality in combination with a direct input paradigm. However, interfaces in the car need to be operable while driving. Distraction, especially visual distraction from the driving task, can lead to critical situations if the sum of attentional demand emerging from both primary and secondary task overextends the available resources. So far, a touchscreen requires a lot of visual attention since its flat surface does not provide any haptic feedback. There have been approaches to make direct touch interaction accessible while driving for simple tasks. Outside the automotive domain, for example in office environments, concepts for sophisticated handling of large displays have already been introduced. Moreover, technological advances lead to new characteristics for interactive surfaces by enabling arbitrary surface shapes. In cars, two main characteristics for upcoming interactive surfaces are largeness and shape. On the one hand, spatial extension is not only increasing through larger displays, but also by taking objects in the surrounding into account for interaction. On the other hand, the flatness inherent in current screens can be overcome by upcoming technologies, and interactive surfaces can therefore provide haptically distinguishable surfaces. This thesis describes the systematic exploration of large and shaped interactive surfaces and analyzes their potential for interaction while driving. Therefore, different prototypes for each characteristic have been developed and evaluated in test settings suitable for their maturity level. Those prototypes were used to obtain subjective user feedback and objective data, to investigate effects on driving and glance behavior as well as usability and user experience. As a contribution, this thesis provides an analysis of the development of interactive surfaces in the car. Two characteristics, largeness and shape, are identified that can improve the interaction compared to conventional touchscreens. The presented studies show that large interactive surfaces can provide new and improved ways of interaction both in driver-only and driver-passenger situations. Furthermore, studies indicate a positive effect on visual distraction when additional static haptic feedback is provided by shaped interactive surfaces. Overall, various, non-exclusively applicable, interaction concepts prove the potential of interactive surfaces for the use in automotive cockpits, which is expected to be beneficial also in further environments where visual attention needs to be focused on additional tasks.Der Einsatz von interaktiven Oberflächen weitet sich mehr und mehr auf die unterschiedlichsten Lebensbereiche aus. Damit sind sie eine mögliche Ausprägung von Mark Weisers Vision der allgegenwärtigen Computer, die aus unserer direkten Wahrnehmung verschwinden. Bei einer Vielzahl von technischen Geräten des täglichen Lebens, wie Smartphones, Tablets oder interaktiven Tischen, sind berührungsempfindliche Oberflächen bereits heute in Benutzung. Schon seit vielen Jahren arbeiten Forscher an einer Weiterentwicklung der Technik, um ihre Vorteile auch in anderen Bereichen, wie beispielsweise der Interaktion zwischen Mensch und Automobil, nutzbar zu machen. Und das mit Erfolg: Interaktive Benutzeroberflächen werden mittlerweile serienmäßig in vielen Fahrzeugen eingesetzt. Der Einbau von immer größeren, in das Cockpit integrierten Touchscreens in Konzeptfahrzeuge zeigt, dass sich diese Entwicklung weiter in vollem Gange befindet. Interaktive Oberflächen ermöglichen das flexible Anzeigen von kontextsensitiven Inhalten und machen eine direkte Interaktion mit den Bildschirminhalten möglich. Auf diese Weise erfüllen sie die sich wandelnden Informations- und Interaktionsbedürfnisse in besonderem Maße. Beim Einsatz von Bedienschnittstellen im Fahrzeug ist die gefahrlose Benutzbarkeit während der Fahrt von besonderer Bedeutung. Insbesondere visuelle Ablenkung von der Fahraufgabe kann zu kritischen Situationen führen, wenn Primär- und Sekundäraufgaben mehr als die insgesamt verfügbare Aufmerksamkeit des Fahrers beanspruchen. Herkömmliche Touchscreens stellen dem Fahrer bisher lediglich eine flache Oberfläche bereit, die keinerlei haptische Rückmeldung bietet, weshalb deren Bedienung besonders viel visuelle Aufmerksamkeit erfordert. Verschiedene Ansätze ermöglichen dem Fahrer, direkte Touchinteraktion für einfache Aufgaben während der Fahrt zu nutzen. Außerhalb der Automobilindustrie, zum Beispiel für Büroarbeitsplätze, wurden bereits verschiedene Konzepte für eine komplexere Bedienung großer Bildschirme vorgestellt. Darüber hinaus führt der technologische Fortschritt zu neuen möglichen Ausprägungen interaktiver Oberflächen und erlaubt, diese beliebig zu formen. Für die nächste Generation von interaktiven Oberflächen im Fahrzeug wird vor allem an der Modifikation der Kategorien Größe und Form gearbeitet. Die Bedienschnittstelle wird nicht nur durch größere Bildschirme erweitert, sondern auch dadurch, dass Objekte wie Dekorleisten in die Interaktion einbezogen werden können. Andererseits heben aktuelle Technologieentwicklungen die Restriktion auf flache Oberflächen auf, so dass Touchscreens künftig ertastbare Strukturen aufweisen können. Diese Dissertation beschreibt die systematische Untersuchung großer und nicht-flacher interaktiver Oberflächen und analysiert ihr Potential für die Interaktion während der Fahrt. Dazu wurden für jede Charakteristik verschiedene Prototypen entwickelt und in Testumgebungen entsprechend ihres Reifegrads evaluiert. Auf diese Weise konnten subjektives Nutzerfeedback und objektive Daten erhoben, und die Effekte auf Fahr- und Blickverhalten sowie Nutzbarkeit untersucht werden. Diese Dissertation leistet den Beitrag einer Analyse der Entwicklung von interaktiven Oberflächen im Automobilbereich. Weiterhin werden die Aspekte Größe und Form untersucht, um mit ihrer Hilfe die Interaktion im Vergleich zu herkömmlichen Touchscreens zu verbessern. Die durchgeführten Studien belegen, dass große Flächen neue und verbesserte Bedienmöglichkeiten bieten können. Außerdem zeigt sich ein positiver Effekt auf die visuelle Ablenkung, wenn zusätzliches statisches, haptisches Feedback durch nicht-flache Oberflächen bereitgestellt wird. Zusammenfassend zeigen verschiedene, untereinander kombinierbare Interaktionskonzepte das Potential interaktiver Oberflächen für den automotiven Einsatz. Zudem können die Ergebnisse auch in anderen Bereichen Anwendung finden, in denen visuelle Aufmerksamkeit für andere Aufgaben benötigt wird