User experience, performance, and social acceptability: usable multimodal mobile interaction

This thesis explores the social acceptability of multimodal interaction in public places with respect to acceptance, adoption and appropriation. Previous work in multimodal interaction has mainly focused on recognition and detection issues without thoroughly considering the willingness of users to adopt these kinds of interactions in their everyday lives. This thesis presents a novel approach to user experience that is theoretically motivated by phenomenology, practiced with mixed-methods, and analysed based on dramaturgical metaphors. In order to explore the acceptance of multimodal interfaces, this thesis presents three studies that look at users’ initial reactions to multimodal interaction techniques: a survey study focusing on gestures, an on-the-street user study, and a follow-up survey study looking at gesture and voice-based interaction. The investigation of multimodal interaction adoption is explored through two studies: an in situ user study of a performative interface and a focus group study using experience prototypes. This thesis explores the appropriation of multimodal interaction by demonstrating the complete design process of a multimodal interface using the performative approach to user experience presented in this thesis. Chapter 3 looks at users’ initial reactions to and acceptance of multimodal interactions. The results of the first survey explored location and audience as factors the influence how individuals behave in public places. Participants in the on-the-street study described the desirable visual aspects of the gestures as playful, cool, or embarrassing aspects of interaction and how gestures could be hidden as everyday actions. These results begin to explain why users accepted or rejected the gestures from the first survey. The second survey demonstrated that the presence of familiar spectators made interaction significantly more acceptable. This result indicates that performative interaction could be made more acceptable by interfaces that support collaborative or social interaction. Chapter 4 explores how users place interactions into a usability context for use in real world settings. In the first user study, participants took advantage of the wide variety of possible performances, and created a wide variety of input, from highly performative to hidden actions, based on location. The ability of this interface to support flexible interactions allowed users to demonstrate the the purposed of their actions differently based on the immediately co-located spectators. Participants in the focus group study discussed how they would go about placing multimodal interactions into real world contexts, using three approaches: relationship to the device, personal meaning, and relationship to functionality. These results demonstrate how users view interaction within a usability context and how that might affect social acceptability. Chapter 5 examines appropriation of multimodal interaction through the completion of an entire design process. The results of an initial survey were used as a baseline of comparison from which to design the following focus group study. Participants in the focus groups had similar motives for accepting multimodal interactions, although the ways in which these were expressed resulted in very different preferences. The desire to use technology in a comfortable and satisfying way meant different things in these different settings. During the ‘in the wild’ user study, participants adapted performance in order to make interaction acceptable in different contexts. In some cases, performance was hidden in public places or shared with familiar spectators in order to successfully incorporate interaction into public places

Mobile phones interaction techniques for second economy people

Second economy people in developing countries are people living in communities that are underserved in terms of basic amenities and social services. Due to literacy challenges and user accessibility problems in rural communities, it is often difficult to design user interfaces that conform to the capabilities and cultural experiences of low-literacy rural community users. Rural community users are technologically illiterate and lack the knowledge of the potential of information and communication technologies. In order to embrace new technology, users will need to perceive the user interface and application as useful and easy to interact with. This requires proper understanding of the users and their socio-cultural environment. This will enable the interfaces and interactions to conform to their behaviours, motivations as well as cultural experiences and preferences and thus enhance usability and user experience. Mobile phones have the potential to increase access to information and provide a platform for economic development in rural communities. Rural communities have economic potential in terms of agriculture and micro-enterprises. Information technology can be used to enhance socio-economic activities and improve rural livelihood. We conducted a study to design user interfaces for a mobile commerce application for micro-entrepreneurs in a rural community in South Africa. The aim of the study was to design mobile interfaces and interaction techniques that are easy to use and meet the cultural preferences and experiences of users who have little to no previous experience of mobile commerce technology. And also to explore the potentials of information technologies rural community users, and bring mobile added value services to rural micro-entrepreneurs. We applied a user-centred design approach in Dwesa community and used qualitative and quantitative research methods to collect data for the design of the user interfaces (graphic user interface and voice user interface) and mobile commerce application. We identified and used several interface elements to design and finally evaluate the graphical user interface. The statistics analysis of the evaluation results show that the users in the community have positive perception of the usefulness of the application, the ease of use and intention to use the application. Community users with no prior experience with this technology were able to learn and understand the interface, recorded minimum errors and a high level of v precision during task performance when they interacted with the shop-owner graphic user interface. The voice user interface designed in this study consists of two flavours (dual tone multi-frequency input and voice input) for rural users. The evaluation results show that community users recorded higher tasks successes and minimum errors with the dual tone multi-frequency input interface than the voice only input interface. Also, a higher percentage of users prefer the dual tone multi-frequency input interface. The t-Test statistical analysis performed on the tasks completion times and error rate show that there was significant statistical difference between the dual tone multi-frequency input interface and the voice input interface. The interfaces were easy to learn, understand and use. Properly designed user interfaces that meet the experience and capabilities of low-literacy users in rural areas will improve usability and users‟ experiences. Adaptation of interfaces to users‟ culture and preferences will enhance information services accessibility among different user groups in different regions. This will promote technology acceptance in rural communities for socio-economic benefits. The user interfaces presented in this study can be adapted to different cultures to provide similar services for marginalised communities in developing countrie

Conceptual model for usable multi-modal mobile assistance during Umrah

Performing Umrah is very demanding and to be performed in very crowded environments. In response to that, many efforts have been initiated to overcome the difficulties faced by pilgrims. However, those efforts focus on acquiring initial perspective and background knowledge before going to Mecca. Findings of preliminary study show that those efforts do not support multi-modality for user interaction. Nowadays the computational capabilities in mobile phones enable it to serve people in various aspects of daily life. Consequently, the mobile phone penetration has increased dramatically in the last decade. Hence, this study aims to propose a comprehensive conceptual model for usable multimodal mobile assistance during Umrah called Multi-model Mobile Assistance during Umrah (MMA-U). Thus, four (4) supporting objectives are formulated, and the Design Science Research Methodology has been adopted. For the usability of MMA-U, Systematic Literature Review (SLR) indicates ten (10) attributes: usefulness, errors rate, simplicity, reliability, ease of use, safety, flexibility, accessibility, attitude, and acceptability. Meanwhile, the content and comparative analysis result in five (5) components that construct the conceptual model of MMA-U: structural, content composition, design principles, development approach, technology, and the design and usability theories. Then, the MMA-U has been reviewed and well-accepted by 15 experts. Later, the MMA-U was incorporated into a prototype called Personal Digital Mutawwif (PDM). The PDM was developed for the purpose of user test in the field. The findings indicate that PDM facilitates the execution of Umrah and successfully meet pilgrims’ needs and expectations. Also, the pilgrims were satisfied and felt that they need to have PDM. In fact, they would recommend PDM to their friends, which mean that use of PDM is safe and suitable while performing Umrah. As a conclusion, the theoretical contribution; the conceptual model of MMA-U; provides guidelines for developing multimodal content mobile applications during Umrah

Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces

This paper contributes to a taxonomy of augmented reality and robotics based on a survey of 460 research papers. Augmented and mixed reality (AR/MR) have emerged as a new way to enhance human-robot interaction (HRI) and robotic interfaces (e.g., actuated and shape-changing interfaces). Recently, an increasing number of studies in HCI, HRI, and robotics have demonstrated how AR enables better interactions between people and robots. However, often research remains focused on individual explorations and key design strategies, and research questions are rarely analyzed systematically. In this paper, we synthesize and categorize this research field in the following dimensions: 1) approaches to augmenting reality; 2) characteristics of robots; 3) purposes and benefits; 4) classification of presented information; 5) design components and strategies for visual augmentation; 6) interaction techniques and modalities; 7) application domains; and 8) evaluation strategies. We formulate key challenges and opportunities to guide and inform future research in AR and robotics

Multimodal access to social media services

Tese de mestrado integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto, Microsoft Language Development Center. 201

Flexible photo retrieval (FlexPhoReS) : a prototype for multimodel personal digital photo retrieval

Digital photo technology is developing rapidly and is motivating more people to have large personal collections of digital photos. However, effective and fast retrieval of digital photos is not always easy, especially when the collections grow into thousands. World Wide Web (WWW) is one of the platforms that allows digital photo users to publish a collection of photos in a centralised and organised way. Users typically find their photos by searching or browsing uSing a keyboard and mouse. Also in development at the moment are alternative user interfaces such as graphical user interfaces with speech (S/GUI) and other multimodal user interfaces which offer more flexibility to users. The aim of this research was to design and evaluate a flexible user interface for a web based personal digital photo retrieval system. A model of a flexible photo retrieval system (FlexPhoReS) was developed based on a review of the literature and a small-scale user study. A prototype, based on the model, was built using MATLAB and WWW technology. FlexPhoReS is a web based personal digital photo retrieval prototype that enables digital photo users to . accomplish photo retrieval tasks (browsing, keyword and visual example searching (CBI)) using either mouse and keyboard input modalities or mouse and speech input modalities. An evaluation with 20 digital photo users was conducted using usability testing methods. The result showed that there was a significant difference in search performance between using mouse and keyboard input modalities and using mouse and speech input modalities. On average, the reduction in search performance time due to using mouse and speech input modalities was 37.31%. Participants were also significantly more satisfied with mouse and speech input modalities than with mouse and keyboard input modalities although they felt that both were complementary. This research demonstrated that the prototype was successful in providing a flexible model of the photo retrieval process by offering alternative input modalities through a multimodal user interface in the World Wide Web environment.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

A Survey of Augmented Reality

© 2015 M. Billinghurst, A. Clark, and G. Lee. This survey summarizes almost 50 years of research and development in the field of Augmented Reality (AR). From early research in the 1960's until widespread availability by the 2010's there has been steady progress towards the goal of being able to seamlessly combine real and virtual worlds. We provide an overview of the common definitions of AR, and show how AR fits into taxonomies of other related technologies. A history of important milestones in Augmented Reality is followed by sections on the key enabling technologies of tracking, display and input devices. We also review design guidelines and provide some examples of successful AR applications. Finally, we conclude with a summary of directions for future work and a review of some of the areas that are currently being researched