Ambient Multimodality: an Asset for Developing Universal Access to the Information Society

International audienceThe paper tries to point out the benefits that can be derived from research advances in the implementation of concepts such as ambient intelligence (AmI) and ubiquitous or pervasive computing for promoting Universal Access (UA) to the Information Society, that is, for contributing to enable everybody, especially Physically Disabled (PD) people, to have easy access to all computing resources and information services that the coming worldwide Information Society will soon make available to the general public. Following definitions of basic concepts relating to multimodal interaction, the significant contribution of multimodality to developing UA is briefly argued. Then, a short state of the art in AmI research is presented. In the last section we bring out the potential contribution of advances in AmI research and technology to the improvement of computer access for PD people. This claim is supported by the following observations: (i) most projects aiming at implementing AmI focus on the design of new interaction modalities and flexible multimodal user interfaces which may facilitate PD users' computer access ; (ii) targeted applications will support users in a wide range of daily activities which will be performed simultaneously with supporting computing tasks; therefore, users will be placed in contexts where they will be confronted with similar difficulties to those encountered by PD users; (iii) AmI applications being intended for the general public, a wide range of new interaction devices and flexible processing software will be available, making it possible to provide PD users with human-computer facilities tailored to their specific needs at reasonable expense.

Communication blades: modular communications for tangible and embedded interfaces

Bladed Tiles is a modular hardware toolkit for building tangible and embedded interface devices. It includes “function blades” and “interaction tiles,” which can provide a flexible, inexpensive, open-ended platform for constructing a wide variety of tangible and embedded interfaces. In this paper, we propose Communication Blades. These are a class of electronic modules with varied computational capabilities for interfacing devices built using bladed tiles toolkit and also for interfacing embedded devices as adapters with external communication networks. These blades provide flexibility by offering the ability to select between different communication technologies and connectivity by providing devices with interoperability over different communication mediums. Furthermore, the modular blade architecture allows different types of communication blades to be plugged in on demand. This reduces the need for development and knowledge of communication protocols by the developers, thus abstracting the underlying complexity. My research work includes studying and designing various communication blades i.e. Serial, USB, Bluetooth and Gumstix. It also includes prototyping, testing and implementing the communication blades

Exploring human-object interaction through force vector measurement

Thesis: S.M., Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2019Cataloged from PDF version of thesis.Includes bibliographical references (pages 101-107).I introduce SCALE, a project aiming to further understand Human-Object Interaction through the real-time analysis of force vector signals, which I have defined as "Force-based Interaction" in this thesis. Force conveys fundamental information in Force-based Interaction, including force intensity, its direction, and object weight - information otherwise difficult to be accessed or inferred from other sensing modalities. To explore the design space of force-based interaction, I have developed the SCALE toolkit, which is composed of modularized 3d-axis force sensors and application APIs. In collaboration with big industry companies, this system has been applied to a variety of application domains and settings, including a retail store, a smart home and a farmers market. In this thesis, I have proposed a base system SCALE, and two additional advanced projects titled KI/OSK and DepthTouch, which build upon the SCALE project.by Takatoshi Yoshida.S.M.S.M. Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Science

Perceptible affordances and feedforward for gestural interfaces: Assessing effectiveness of gesture acquisition with unfamiliar interactions

The move towards touch-based interfaces disrupts the established ways in which users manipulate and control graphical user interfaces. The predominant mode of interaction established by the desktop interface is to ‘double-click’ an icon in order to open an application, file or folder. Icons show users where to click and their shape, colour and graphic style suggests how they respond to user action. In sharp contrast, in a touch-based interface, an action may require a user to form a gesture with a certain number of fingers, a particular movement, and in a specific place. Often, none of this is suggested in the interface. This thesis adopts the approach of research through design to address the problem of how to inform the user about which gestures are available in a given touch-based interface, how to perform each gesture, and, finally, the effect of each gesture on the underlying system. Its hypothesis is that presenting automatic and animated visual prompts that depict touch and preview gesture execution will mitigate the problems users encounter when they execute commands within unfamiliar gestural interfaces. Moreover, the thesis claims the need for a new framework to assess the efficiency of gestural UI designs. A significant aspect of this new framework is a rating system that was used to assess distinct phases within the users’ evaluation and execution of a gesture. In order to support the thesis hypothesis, two empirical studies were conducted. The first introduces the visual prompts in support of training participants in unfamiliar gestures and gauges participants’ interpretation of their meaning. The second study consolidates the design features that yielded fewer error rates in the first study and assesses different interaction techniques, such as the moment to display the visual prompt. Both studies demonstrate the benefits in providing visual prompts to improve user awareness of available gestures. In addition, both studies confirm the efficiency of the rating system in identifying the most common problems users have with gestures and identifying possible design features to mitigate such problems. The thesis contributes: 1) a gesture-and-effect model and a corresponding rating system that can be used to assess gestural user interfaces, 2) the identification of common problems users have with unfamiliar gestural interfaces and design recommendations to mitigate these problems, and 3) a novel design technique that will improve user awareness of unfamiliar gestures within novel gestural interfaces