9 research outputs found

    Description des tĂąches avec un systĂšme interactif multiutilisateur et multimodal : Etude comparative de notations

    No full text
    International audienceMulti-user multimodal interactive systems involve multiple users who can use multiple interactionmodalities. Multi-user multimodal systems are becoming more prevalent, especially systems based on largeshared multi-touch surfaces or video game centers such as Wii or Xbox. In this article we address thedescription of the tasks with such interactive systems. We review existing notations for the description of taskswith a multi-user multimodal interactive system and focus particularly on tree-based notations. For elementarytasks (e.g. actions), we also consider the notations that describe multimodal interaction. The contribution isthen a comparison of existing notations based on a set of organized concepts. While some concepts are generalto any notation, other concepts are specific to human-computer interaction, or to multi-user interaction andfinally to multimodal interaction.De nombreux systĂšmes interactifs, professionnels ou grand public, permettent conjointementl’interaction multiutilisateur et multimodale. Un systĂšme interactif est multimodal lorsqu’un utilisateur peutinteragir avec le systĂšme par l’usage de plusieurs modalitĂ©s d’interaction (en entrĂ©e ou en sortie) de façonparallĂšle ou non. Nous constatons que de plus en plus de systĂšmes multiutilisateurs ou collecticiels sontmultimodaux, comme ceux construits autour d’une surface interactive et les consoles de jeu de type Wii ouXbox. Nous traitons dans cet article de la description des tĂąches-utilisateur avec de tels systĂšmes interactifsmultiutilisateurs et multimodaux. PrĂ©cisĂ©ment, nous dressons un panorama des notations existantes permettantla description des tĂąches mono ou multi-utilisateur avec une attention particuliĂšre pour les notations Ă  based’arbre de tĂąches. Nous focalisons aussi sur les tĂąches Ă©lĂ©mentaires ou actions mono/multi-modales del’utilisateur en considĂ©rant les notations de description de l’interaction multimodale. Pour cela, nousproposons une Ă©tude comparative d'un ensemble de notations de description selon une grille d’analyseregroupant des concepts gĂ©nĂ©raux Ă  l’interaction et des concepts propres Ă  l’interaction multiutilisateur etmultimodale

    Bridging the Gap between a Behavioural Formal Description Technique and User Interface description language: Enhancing ICO with a Graphical User Interface markup language

    Get PDF
    International audienceIn the last years, User Interface Description Languages (UIDLs) appeared as a suitable solution for developing interactive systems. In order to implement reliable and efficient applications, we propose to employ a formal description technique called ICO (Interactive Cooperative Object) that has been developed to cope with complex behaviours of interactive systems including event-based and multimodal interactions. So far, ICO is able to describe most of the parts of an interactive system, from functional core concerns to fine grain interaction techniques, but, even if it addresses parts of the rendering, it still not has means to describe the effective rendering of such interactive system. This paper presents a solution to overcome this gap using markup languages. A first technique is based on the Java technology called JavaFX and a second technique is based on the emergent UsiXML language for describing user interface components for multi-target platforms. The proposed approach offers a bridge between markup language based descriptions of the user interface components and a robust technique for describing behaviour using ICO modelling. Furthermore, this paper highlights how it is possible to take advantage from both behavioural and markup language description techniques to propose a new model-based approach for prototyping interactive systems. The proposed approach is fully illustrated by a case study using an interactive application embedded into interactive aircraft cockpits

    Model-Driven Development of Interactive Multimedia Applications

    Get PDF
    The development of highly interactive multimedia applications is still a challenging and complex task. In addition to the application logic, multimedia applications typically provide a sophisticated user interface with integrated media objects. As a consequence, the development process involves different experts for software design, user interface design, and media design. There is still a lack of concepts for a systematic development which integrates these aspects. This thesis provides a model-driven development approach addressing this problem. Therefore it introduces the Multimedia Modeling Language (MML), a visual modeling language supporting a design phase in multimedia application development. The language is oriented on well-established software engineering concepts, like UML 2, and integrates concepts from the areas of multimedia development and model-based user interface development. MML allows the generation of code skeletons from the models. Thereby, the core idea is to generate code skeletons which can be directly processed in multimedia authoring tools. In this way, the strengths of both are combined: Authoring tools are used to perform the creative development tasks while models are used to design the overall application structure and to enable a well-coordinated development process. This is demonstrated using the professional authoring tool Adobe Flash. MML is supported by modeling and code generation tools which have been used to validate the approach over several years in various student projects and teaching courses. Additional prototypes have been developed to demonstrate, e.g., the ability to generate code for different target platforms. Finally, it is discussed how models can contribute in general to a better integration of well-structured software development and creative visual design

    From Metamodeling to Automatic Generation of Multimodal Interfaces for Ambient Computing

    Get PDF
    International audiencehis paper presents our approach to design multichannel and multimodal applications as part of ambient intelligence. Computers are increasingly present in our environments, whether at work (computers, photocopiers), at home (video player, hi-fi, microwave), in our cars, etc. They are more adaptable and context-sensitive (e.g., the car radio that lowers the volume when the mobile phone rings). Unfortunately, while they should provide smart services by combining their skills, they are not yet designed to communicate together. Our results, mainly based on the use of a software bus and a workflow, show that different devices (such as Wiimote, multi-touch screen, telephone, etc.) can be coordinated in order to activate real things (such as lamp, fan, robot, webcam, etc.). A smart digital home case study illustrates how using our approach to design with ease some parts of the ambient system and to redesign them during runtime

    A multi-modelS based approach for the modelling and the analysis of usable and resilient partly autonomous interactive systems

    Get PDF
    La croissance prĂ©visionnelle du trafic aĂ©rien est telle que les moyens de gestion actuels doivent Ă©voluer et ĂȘtre amĂ©liorĂ©s et l'automatisation de certains aspects de cette gestion semble ĂȘtre un moyen pour gĂ©rer cet accroissement du trafic tout en gardant comme invariant un niveau de sĂ©curitĂ© constant. Toutefois, cette augmentation du trafic pourrait entraĂźner un accroissement de la variabilitĂ© de la performance de l'ensemble des moyens de gestion du trafic aĂ©rien, en particulier dans le cas de dĂ©gradation de cette automatisation. Les systĂšmes de gestion du trafic aĂ©rien sont considĂ©rĂ©s comme complexes car ils impliquent de nombreuses interactions entre humains et systĂšmes, et peuvent ĂȘtre profondĂ©ment influencĂ©s par les aspects environnementaux (mĂ©tĂ©orologie, organisation, stress ...) et tombent, de fait, dans la catĂ©gorie des SystĂšmes Sociotechniques (STS) (Emery & Trist, 1960). A cause de leur complexitĂ©, les interactions entre les diffĂ©rents Ă©lĂ©ments (humains, systĂšmes et organisations) de ces STS peuvent ĂȘtre linĂ©aires et partiellement non linĂ©aires, ce qui rend l'Ă©volution de leur performance difficilement prĂ©visible. Au sein de ces STS, les systĂšmes interactifs doivent ĂȘtre utilisables, i.e. permettre Ă  leurs utilisateurs d'accomplir leurs tĂąches de maniĂšre efficace et efficiente. Un STS doit aussi ĂȘtre rĂ©silient aux perturbations telles que les dĂ©faillances logicielles et matĂ©rielles, les potentielles dĂ©gradations de l'automatisation ou les problĂšmes d'interaction entre les systĂšmes et leurs opĂ©rateurs. Ces problĂšmes peuvent affecter plusieurs aspects des systĂšmes sociotechniques comme les ressources, le temps d'exĂ©cution d'une tĂąche, la capacitĂ© Ă  d'adaptation Ă  l'environnement... Afin de pouvoir analyser l'impact de ces perturbations et d'Ă©valuer la variabilitĂ© de la performance d'un STS, des techniques et mĂ©thodes dĂ©diĂ©es sont requises. Elles doivent fournir un support Ă  la modĂ©lisation et Ă  l'analyse systĂ©matique de l'utilisabilitĂ© et de la rĂ©silience de systĂšmes interactifs aux comportements partiellement autonomes. Elles doivent aussi permettre de dĂ©crire et de structurer un grand nombre d'informations, ainsi que de traiter la variabilitĂ© de chaque Ă©lĂ©ment du STS et la variabilitĂ© liĂ©e Ă  leurs interrelations. Les techniques et mĂ©thodes existantes ne permettent actuellement ni de modĂ©liser un STS dans son ensemble, ni d'en analyser les propriĂ©tĂ©s d'utilisabilitĂ© et de rĂ©silience (ou alors se focalisent sur un sous-ensemble du STS perdant, de fait, la vision systĂ©mique). Enfin, elles ne fournissent pas les moyens d'analyser la migration de tĂąches suite Ă  l'introduction d'une nouvelle technologie ou d'analyser la variabilitĂ© de la performance en cas de dĂ©gradation de fonctions rĂ©cemment automatisĂ©es. Ces arguments sont dĂ©veloppĂ©s dans la thĂšse et appuyĂ©s par une analyse dĂ©taillĂ©e des techniques de modĂ©lisation existantes et des mĂ©thodes qui leurs sont associĂ©es. La contribution prĂ©sentĂ©e est basĂ©e sur l'identification d'un ensemble d'exigences requises pour pouvoir modĂ©liser et analyser chacun des Ă©lĂ©ments d'un STS. Certaines de ces exigences ont Ă©tĂ© remplies grĂące Ă  l'utilisation de techniques de modĂ©lisation existantes, les autres grĂące Ă  l'extension et au raffinement d'autres techniques. Cette thĂšse propose une approche qui intĂšgre 3 techniques en particulier : FRAM (centrĂ©e sur les fonctions organisationnelles), HAMSTERS (centrĂ©e les objectifs et activitĂ©s humaines) et ICO (dĂ©diĂ©e Ă  la modĂ©lisation du comportement des systĂšmes interactifs). Cette approche est illustrĂ©e par un exemple mettant en Ɠuvre les extensions proposĂ©es et l'intĂ©gration des modĂšles. Une Ă©tude de cas plus complexe sur la gestion du trafic aĂ©rien (changement de route d'un avion en cas de mauvaises conditions mĂ©tĂ©orologiques) est ensuite prĂ©sentĂ©e pour montrer le passage Ă  l'Ă©chelle de l'approche. Elle met en avant les bĂ©nĂ©fices de l'intĂ©gration des modĂšles pour la prise en compte de la variabilitĂ© de la performance des diffĂ©rents Ă©lĂ©ments d'un STSThe current European Air Traffic Management (ATM) System needs to be improved for coping with the growth in air traffic forecasted for next years. It has been broadly recognised that the future ATM capacity and safety objectives can only be achieved by an intense enhancement of integrated automation support. However, increase of automation might come along with an increase of performance variability of the whole ATM System especially in case of automation degradation. ATM systems are considered complex as they encompass interactions involving humans and machines deeply influenced by environmental aspects (i.e. weather, organizational structure) making them belong to the class of Socio-Technical Systems (STS) (Emery & Trist, 1960). Due to this complexity, the interactions between the STS elements (human, system and organisational) can be partly linear and partly non-linear making its performance evolution complex and hardly predictable. Within such STS, interactive systems have to be usable i.e. enabling users to perform their tasks efficiently and effectively while ensuring a certain level of operator satisfaction. Besides, the STS has to be resilient to adverse events including potential automation degradation issues but also interaction problems between their interactive systems and the operators. These issues may affect several STS aspects such as resources, time in tasks performance, ability to adjust to environment, etc. In order to be able to analyse the impact of these perturbations and to assess the potential performance variability of a STS, dedicated techniques and methods are required. These techniques and methods have to provide support for modelling and analysing in a systematic way usability and resilience of interactive systems featuring partly autonomous behaviours. They also have to provide support for describing and structuring a large amount of information and to be able to address the variability of each of STS elements as well as the variability related to their interrelations. Current techniques, methods and processes do not enable to model a STS as a whole and to analyse both usability and resilience properties. Also, they do not embed all the elements that are required to describe and analyse each part of the STS (such as knowledge of different types which is needed by a user for accomplishing tasks or for interacting with dedicated technologies). Lastly, they do not provide means for analysing task migrations when a new technology is introduced or for analysing performance variability in case of degradation of the newly introduced automation. Such statements are argued in this thesis by a detailed analysis of existing modelling techniques and associated methods highlighting their advantages and limitations. This thesis proposes a multi-models based approach for the modelling and the analysis of partly-autonomous interactive systems for assessing their resilience and usability. The contribution is based on the identification of a set of requirements needed being able to model and analyse each of the STS elements. Some of these requirements were met by existing modelling techniques, others were reachable by extending and refining existing ones. This thesis proposes an approach which integrates 3 modelling techniques: FRAM (focused on organisational functions), HAMSTERS (centred on human goals and activities) and ICO (dedicated to the modelling of interactive systems). The principles of the multi-models approach is illustrated on an example for carefully showing the extensions proposed to the selected modelling techniques and how they integrate together. A more complex case study from the ATM World is then presented to demonstrate the scalability of the approach. This case study, dealing with aircraft route change due to bad weather conditions, highlights the ability of the integration of models to cope with performance variability of the various parts of the ST

    A Scholarship Approach to Model-Driven Engineering

    Get PDF
    Model-Driven Engineering is a paradigm for software engineering where software models are the primary artefacts throughout the software life-cycle. The aim is to define suitable representations and processes that enable precise and efficient specification, development and analysis of software. Our contributions to Model-Driven Engineering are structured according to Boyer\u27s four functions of academic activity - the scholarships of teaching, discovery, application and integration. The scholarships share a systematic approach towards seeking new insights and promoting progressive change. Even if the scholarships have their differences they are compatible so that theory, practice and teaching can strengthen each other.Scholarship of Teaching: While teaching Model-Driven Engineering to under-graduate students we introduced two changes to our course. The first change was to introduce a new modelling tool that enabled the execution of software models while the second change was to adapt pair lecturing to encourage the students to actively participate in developing models during lectures. Scholarship of Discovery: By using an existing technology for transforming models into source code we translated class diagrams and high-level action languages into natural language texts. The benefit of our approach is that the translations are applicable to a family of models while the texts are reusable across different low-level representations of the same model.Scholarship of Application: Raising the level of abstraction through models might seem a technical issue but our collaboration with industry details how the success of adopting Model-Driven Engineering depends on organisational and social factors as well as technical. Scholarship of Integration: Building on our insights from the scholarships above and a study at three large companies we show how Model-Driven Engineering empowers new user groups to become software developers but also how engineers can feel isolated due to poor tool support. Our contributions also detail how modelling enables a more agile development process as well as how the validation of models can be facilitated through text generation.The four scholarships allow for different possibilities for insights and explore Model-Driven Engineering from diverse perspectives. As a consequence, we investigate the social, organisational and technological factors of Model-Driven Engineering but also examine the possibilities and challenges of Model-Driven Engineering across disciplines and scholarships

    Modellierung und Generierung von BenutzeroberflĂ€chen fĂŒr interaktive Softwaresysteme unter der Nutzung von Mustern

    Get PDF
    Die Dissertation umfasst den Entwurf einer Entwicklungsumgebung fĂŒr BenutzeroberflĂ€chen, der modell- und musterbasierte UI-Entwicklung zu einem integrierten Entwicklungs- und Generierungsansatz kombiniert. Die Grundlagen der modell- und musterbasierten Entwicklung werden dargestellt und existierende Implementierungen analysiert. Die Ergebnisse der Untersuchungen fließen in den Entwurf der kombinierten Entwicklungsumgebung, der detailliert beschrieben wird, ein. Die Arbeit beinhaltet eine Fallstudie, die die praktische Bedeutung der theoretischen AusfĂŒhrungen verdeutlicht

    Analysis of the Software Architectural Support of the Usability of Mobile Applications

    Get PDF
    Rasante Innovationszyklen, kurze ProdukteinfĂŒhrungszeiten und ein hoher Konkurrenzdruck sind typische Rahmenbedingungen fĂŒr die Entwicklung mobiler Anwendungen. Dies sind Anwendungen, die auf mobilen EndgerĂ€ten laufen und in verschiedenen Umgebungen verwendet werden. Usability (Benutzbarkeit) kann durch die Softwarearchitektur einer Anwendung unterstĂŒtzt, aber auch behindert werden. Je spĂ€ter im Softwareentwicklungsprozess Usability beachtet wird, umso aufwendiger werden Änderungen an der Softwarearchitektur. Um dieses Risiko zu verringern, ist es nötig, so frĂŒh wie möglich offenzulegen, ob Usability-Anforderungen architektonisch unterstĂŒtzt werden. Potenziell hohe ArchitekturĂ€nderungen werden mit Methoden zur szenario-basierten Softwarearchitekturanalyse ermittelt; es wird dabei verifiziert, dass ein QualitĂ€tsmerkmal architektonisch berĂŒcksichtigt wurde. FrĂŒhere Methoden bezĂŒglich Usability erreichen dieses Ziel, erscheinen aber sehr komplex: Sie erfordern zum einen Wissen ĂŒber Patterns und zum anderen sind die Freiheitsgrade beim Erstellen, AuswĂ€hlen und Evaluieren von Szenarios hoch. Wie die frĂŒheren Methoden Usability-Attribute verwenden, behindert zudem eine engere Kooperation mit dem Usability Engineering, fĂŒr das gemeinsame Begriffe und Vorgehensweisen grundlegend wĂ€ren. Deshalb ist es notwendig, eine Methode zu konstruieren, die einfacher und interdisziplinĂ€rer ausgerichtet ist. Aufgrund dessen werden in dieser Forschungsarbeit mittels Literaturstudien zuerst Forschungsfragen, dann Hilfsmittel und schließlich eine theoretisch fundierte Methode erarbeitet. Um diese zu validieren und zu vereinfachen, durchlĂ€uft sie - mit kanonischer anwendungsnaher Forschung - zwei Fallstudien zu mobilen Anwendungen. Ergebnis ist die szenario-basierte Methode SATURN („SoftwareArchitekTuranalyse von Usability-anfoRderungeN“), in der anfangs mittels Nutzungskontextanalyse die Interaktionsszenarios erstellt werden, die fĂŒr Anwender relevant sind. Hilfsmittel umfassen die Faktoren des mobilen Nutzungskontexts und einen Katalog von 50 potenziell architektursensitiven Interaktionsszenarios. Diese sind von Patterns abgeleitet, referenzieren sie und unterliegen einem definierten Lebenszyklus. Die Analyse stĂŒtzt sich auf die verwendete Architekturdefinition und auf das Prinzip der Sichtenmodelle. Bewertet wird, inwiefern Struktur oder Verhalten von architektonischen Elementen verhindern können, dass ein Interaktionsszenario (hypothetisch) durchgefĂŒhrt werden kann. Betrachtet wird dabei, wie Usability berĂŒcksichtigt wurde, welche vor- und nachteiligen Architekturentscheidungen und welche Austauschbeziehungen mit anderen QualitĂ€tsmerkmalen bestehen. Die Ergebnisse von SATURN fließen zurĂŒck zur Erstellung der Softwarearchitektur und zum Usability Engineering. Die Methode ist auch mit einem Nutzertest kombinierbar. Mit SATURN ist die Analyse der architektonischen UnterstĂŒtzung fĂŒr die Usability mobiler Anwendungen einfacher als mit frĂŒheren Arbeiten. Dies inspiriert zu weiterer Forschung, wie beispielsweise Fallstudien zum Zusammenhang zwischen Usability und Softwarearchitektur, die Ausrichtung der Methode auf andere QualitĂ€tsmerkmale, neue konstruktive Möglichkeiten in agilen Prozessen oder allgemein die Koordination von Usability Engineering und Softwareentwicklung.The software development of mobile applications, i.e. applications which run on mobile devices and are used in various environments, faces a fast time to market, high competitive pressure, short technical innovation cycles, and high user expectations regarding usability. The software architecture of an application can support but also constrain usability. The later in software development usability is considered, the costlier architectural modifications become. In order to reduce this risk, it is necessary to discover usability requirements that are not supported architecturally as early as possible. Potentially high architectural changes are elicited using scenario-based software architecture analyses; it is verified, that a quality factor was considered architecturally. Earlier works regarding usability achieve this, though they appear very complex: they depend on the knowledge of patterns and leave room for interpretation while creating, selecting and evaluating scenarios. Also, respective uses of usability attributes hinder a further cooperation with usability engineering, because common terms and methods are a prerequisite for this. Therefore, it is necessary to construct a new method which is easier to use and more interdisciplinary-oriented. In order to describe research questions, means for the method and a first version of the method itself, literature studies are conducted. Afterwards, the method is validated and improved through two case studies with mobile applications adhering to canonical action research. In the new scenario-based method SATURN („Software ArchitecTure analysis of Usability-RequiremeNts“), interaction scenarios that are relevant from a users’ point of view are depicted and selected based on a usage context analysis. This is facilitated by providing factors of the mobile usage context and a catalog of 50 interaction scenarios that are potentially architectural sensitive (derived from patterns, referring to them, and complying to a scenario life cycle). The analysis of the architecture itself is based on the principle of view models. We analyze, in what way structure or behavior of architectural elements can constrain the interaction described by a specific interaction scenario. Results verify that usability was considered architecturally, explain disadvantageous and advantageous architectural decisions as well as trade-offs with other quality factors. Thus, results can flow back to architectural design and usability engineering. Additionally, the method is combined with a user test. With this research, analyzing and assessing the architectural support for the usability of mobile applications is easier than with earlier works. This inspires further research, for example, more case studies regarding the relationship between usability and software architecture, other quality factors and software architecture, new possibilities for construction in agile processes, and the cooperation amongst the fields usability engineering and software engineering in general
    corecore