Multimodal Shared-Control Interaction for Mobile Robots in AAL Environments

This dissertation investigates the design, development and implementation of cognitively adequate, safe and robust, spatially-related, multimodal interaction between human operators and mobile robots in Ambient Assisted Living environments both from the theoretical and practical perspectives. By focusing on different aspects of the concept Interaction, the essential contribution of this dissertation is divided into three main research packages; namely, Formal Interaction, Spatial Interaction and Multimodal Interaction in AAL. As the principle package, in Formal Interaction, research effort is dedicated to developing a formal language based interaction modelling and management solution process and a unified dialogue modelling approach. This package aims to enable a robust, flexible, and context-sensitive, yet formally controllable and tractable interaction. This type of interaction can be used to support the interaction management of any complex interactive systems, including the ones covered in the other two research packages. In the second research package, Spatial Interaction, a general qualitative spatial knowledge based multi-level conceptual model is developed and proposed. The goal is to support a spatially-related interaction in human-robot collaborative navigation. With a model-based computational framework, the proposed conceptual model has been implemented and integrated into a practical interactive system which has been evaluated by empirical studies. It has been particularly tested with respect to a set of high-level and model-based conceptual strategies for resolving the frequent spatially-related communication problems in human-robot interaction. Last but not least, in Multimodal Interaction in AAL, attention is drawn to design, development and implementation of multimodal interaction for elderly persons. In this elderly-friendly scenario, ageing-related characteristics are carefully considered for an effective and efficient interaction. Moreover, a standard model based empirical framework for evaluating multimodal interaction is provided. This framework was especially applied to evaluate a minutely developed and systematically improved elderly-friendly multimodal interactive system through a series of empirical studies with groups of elderly persons

Design and evaluation of auditory spatial cues for decision making within a game environment for persons with visual impairments

An audio platform game was created and evaluated in order to answer the question of whether or not an audio game could be designed that effectively conveys the spatial information necessary for persons with visual impairments to successfully navigate the game levels and respond to audio cues in time to avoid obstacles. The game used several types of audio cues (sounds and speech) to convey the spatial setup (map) of the game world. Most audio-only players seemed to be able to create a workable mental map from the game\u27s sound cues alone, pointing to potential for the further development of similar audio games for persons with visual impairments. The research also investigated the navigational strategies used by persons with visual impairments and the accuracy of the participants\u27 mental maps as a consequence of their navigational strategy. A comparisons of the maps created by visually impaired participants with those created by sighted participants playing the game with and without graphics, showed no statistically significant difference in map accuracy between groups. However, there was a marked difference between the number of invented objects when we compared this value between the sighted audio-only group and the other groups, which could serve as an area for future research

Integrating Haptic Feedback into Mobile Location Based Services

Haptics is a feedback technology that takes advantage of the human sense of touch by applying forces, vibrations, and/or motions to a haptic-enabled device such as a mobile phone. Historically, human-computer interaction has been visual - text and images on the screen. Haptic feedback can be an important additional method especially in Mobile Location Based Services such as knowledge discovery, pedestrian navigation and notification systems. A knowledge discovery system called the Haptic GeoWand is a low interaction system that allows users to query geo-tagged data around them by using a point-and-scan technique with their mobile device. Haptic Pedestrian is a navigation system for walkers. Four prototypes have been developed classified according to the user’s guidance requirements, the user type (based on spatial skills), and overall system complexity. Haptic Transit is a notification system that provides spatial information to the users of public transport. In all these systems, haptic feedback is used to convey information about location, orientation, density and distance by use of the vibration alarm with varying frequencies and patterns to help understand the physical environment. Trials elicited positive responses from the users who see benefit in being provided with a “heads up” approach to mobile navigation. Results from a memory recall test show that the users of haptic feedback for navigation had better memory recall of the region traversed than the users of landmark images. Haptics integrated into a multi-modal navigation system provides more usable, less distracting but more effective interaction than conventional systems. Enhancements to the current work could include integration of contextual information, detailed large-scale user trials and the exploration of using haptics within confined indoor spaces

The influence of off-screen landmarks on user orientation

Dissertation submitted in partial fulfillment of the requirements for the Degree of Master of Science in Geospatial Technologies.Maps displayed on hand held devices, such as smartphones; provide limited visibility due to their small screen display. In order to overcome display limitations, researchers have developed new techniques that point users to objects and landmarks located off-screen, which is known as an “off-screen interface”. Since this is a new type of map interface, there is little information regarding the effect it has on the user’s perception of the environment and spatial orientation abilities. This thesis investigates the relationship of a map display to the user’s orientation in the environment. A field experiment was conducted in an urban environment, where twenty-four participants were individually guided along an unfamiliar route. All participants were provided with a map displayed on a smartphone, whereas only half were additionally equipped with the off-screen interface. Tasks involving user’s orientation knowledge were preformed along the route and at the end point. In addition, user’s interaction with the map was monitored. Results were compared between the two interface groups and individual difference, such as gender and sense of direction abilities, were taken into account. The results do not indicate a significant difference between the two interfaces in relocation to orientation abilities. However, considerable differences among the interface groups were found with relation to gender and senses of direction. Moreover, a significant difference was found between groups in relation to user-map interaction. Based on the results of this study, a relation between the off-screen map interface and its user’s orientation abilities can be observed

Timing of Pedestrian Navigation Instructions

During pedestrian navigation in outdoor urban environments we often utilize assistance systems to support decision-making. These systems help wayfinders by providing relevant information withing the context of their surroundings, e.g., landmark-based instructions of the type "turn left at the church". Next to the instruction type and content, also the timing of the instruction must be considered in order to facilitate the wayfinding process. In this work we present our findings concerning the user and environmental factors that have an impact on the timing of instructions. We applied a survival analysis on data collected through an experiment in a realistic virtual environment in order to analyze the expected distance to the decision point until instructions are needed. The presented results can be used by navigation systems for instruction timing based on the characteristics of the current wayfinder and environment

Gender differences in navigation dialogues with computer systems

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Gender is among the most influential of the factors underlying differences in spatial abilities, human communication and interactions with and through computers. Past research has offered important insights into gender differences in navigation and language use. Yet, given the multidimensionality of these domains, many issues remain contentious while others unexplored. Moreover, having been derived from non-interactive, and often artificial, studies, the generalisability of this research to interactive contexts of use, particularly in the practical domain of Human-Computer Interaction (HCI), may be problematic. At the same time, little is known about how gender strategies, behaviours and preferences interact with the features of technology in various domains of HCI, including collaborative systems and systems with natural language interfaces. Targeting these knowledge gaps, the thesis aims to address the central question of how gender differences emerge and operate in spatial navigation dialogues with computer systems. To this end, an empirical study is undertaken, in which, mixed-gender and same-gender pairs communicate to complete an urban navigation task, with one of the participants being under the impression that he/she interacts with a robot. Performance and dialogue data were collected using a custom system that supported synchronous navigation and communication between the user and the robot. Based on this empirical data, the thesis describes the key role of the interaction of gender in navigation performance and communication processes, which outweighed the effect of individual gender, moderating gender differences and reversing predicted patterns of performance and language use. This thesis has produced several contributions; theoretical, methodological and practical. From a theoretical perspective, it offers novel findings in gender differences in navigation and communication. The methodological contribution concerns the successful application of dialogue as a naturalistic, and yet experimentally sound, research paradigm to study gender and spatial language. The practical contributions include concrete design guidelines for natural language systems and implications for the development of gender-neutral interfaces in specific domains of HCI

Schematisation in Hard-copy Tactile Orientation Maps

This dissertation investigates schematisation of computer-generated tactile orientation maps that support mediation of spatial knowledge of unknown urban environments. Computergenerated tactile orientation maps are designed to provide the blind with an overall impression of their surroundings. Their details are displayed by means of elevated features that are created by embossers and can be distinguished by touch. The initial observation of this dissertation states that only very little information is actually transported through tactile maps owing to the coarse resolution of tactual senses and the cognitive effort involved in the serial exploration of tactile maps. However, the differences between computer-generated, embossed tactile maps and manufactured, deep-drawn tactile maps are significant. Therefore the possibilities and confines of communicating information through tactile maps produced with embossers is a primary area of research. This dissertation has been able to demonstrate that the quality of embossed prints is an almost equal alternative to traditionally manufactured deep-drawn maps. Their great advantage is fast and individual production and (apart from the initial procurement costs for the printer)low price, accessibility and easy understanding without the need of prior time-consuming training. Simplification of tactile maps is essential, even more so than in other maps. It can be achieved by selecting a limited number from all map elements available. Qualitative simplification through schematisation may present an additional option to simplification through quantitative selection. In this context schematisation is understood as cognitively motivated simplification of geometry and synchronised maintenance of topology. Rather than further reducing the number of displayed objects, the investigation concentrates on how the presentation of different forms of streets (natural vs. straightened) and junctions (natural vs. prototypical) affects the transfer of knowledge. In a second area of research, a thesis establishes that qualitative simplification of tactile orientation maps through schematisation can enhance their usability and make them easier to understand than maps that have not been schematised. The dissertation shows that simplifying street forms and limiting them to prototypical junctions does not only accelerate map exploration but also has a beneficial influence on retention performance. The majority of participants that took part in the investigation selected a combination of both as their preferred display option. Tactile maps that have to be tediously explored through touch, uncovering every detail, complicate attaining a first impression or an overall perception. A third area of research is examined, establishing which means could facilitate map readersâ options to discover certain objects on the map quickly and without possessing a complete overview. Three types of aids are examined: guiding lines leading from the frame of the map to the object, position indicators represented by position markers at the frame of the map and coordinate specifications found within a grid on the map. The dissertation shows that all three varieties can be realised by embossers. Although a guiding line proves to be fast in size A4 tactile maps containing only one target object and few distracting objects, it also impedes further exploration of the map (similar to the grid). In the following, advantages and drawbacks of the various aids in this and other applications are discussed. In conclusion the dissertation elaborates on the linking points of all three examinations. They connect and it is argued that cognitively motivated simplification should be a principle of construction for embossed tactile orientation maps in order to support their use and comprehension. A summary establishes the recommendations that result from this dissertation regarding construction of tactile orientation maps considering the limitations through embosser constraints. Then I deliberate how to adapt schematisation of other maps contingent to intended function, previous knowledge of the map reader, and the relation between the time in which knowledge is acquired and the time it is employed. Closing the dissertation, I provide an insight into its confines and deductions and finish with a prospective view to possible transfers of the findings to other applications, e.g. multimedia or interactive maps on pin-matrix displays and devices