53 research outputs found
Hand Gesture Interaction with Human-Computer
Hand gestures are an important modality for human computer interaction. Compared to many existing interfaces, hand gestures have the advantages of being easy to use, natural, and intuitive. Successful applications of hand gesture recognition include computer games control, human-robot interaction, and sign language recognition, to name a few. Vision-based recognition systems can give computers the capability of understanding and responding to hand gestures. The paper gives an overview of the field of hand gesture interaction with Human- Computer, and describes the early stages of a project about gestural command sets, an issue that has often been neglected. Currently we have built a first prototype for exploring the use of pieand marking menus in gesture-based interaction. The purpose is to study if such menus, with practice, could support the development of autonomous gestural command sets. The scenario is remote control of home appliances, such as TV sets and DVD players, which in the future could be extended to the more general scenario of ubiquitous computing in everyday situations. Some early observations are reported, mainly concerning problems with user fatigue and precision of gestures. Future work is discussed, such as introducing flow menus for reducing fatigue, and control menus for continuous control functions. The computer vision algorithms will also have to be developed further
Ambient Intelligence for Next-Generation AR
Next-generation augmented reality (AR) promises a high degree of
context-awareness - a detailed knowledge of the environmental, user, social and
system conditions in which an AR experience takes place. This will facilitate
both the closer integration of the real and virtual worlds, and the provision
of context-specific content or adaptations. However, environmental awareness in
particular is challenging to achieve using AR devices alone; not only are these
mobile devices' view of an environment spatially and temporally limited, but
the data obtained by onboard sensors is frequently inaccurate and incomplete.
This, combined with the fact that many aspects of core AR functionality and
user experiences are impacted by properties of the real environment, motivates
the use of ambient IoT devices, wireless sensors and actuators placed in the
surrounding environment, for the measurement and optimization of environment
properties. In this book chapter we categorize and examine the wide variety of
ways in which these IoT sensors and actuators can support or enhance AR
experiences, including quantitative insights and proof-of-concept systems that
will inform the development of future solutions. We outline the challenges and
opportunities associated with several important research directions which must
be addressed to realize the full potential of next-generation AR.Comment: This is a preprint of a book chapter which will appear in the
Springer Handbook of the Metavers
Direct interaction with large displays through monocular computer vision
Large displays are everywhere, and have been shown to provide higher productivity gain and user satisfaction compared to traditional desktop monitors. The computer mouse remains the most common input tool for users to interact with these larger displays. Much effort has been made on making this interaction more natural and more intuitive for the user. The use of computer vision for this purpose has been well researched as it provides freedom and mobility to the user and allows them to interact at a distance. Interaction that relies on monocular computer vision, however, has not been well researched, particularly when used for depth information recovery. This thesis aims to investigate the feasibility of using monocular computer vision to allow bare-hand interaction with large display systems from a distance. By taking into account the location of the user and the interaction area available, a dynamic virtual touchscreen can be estimated between the display and the user. In the process, theories and techniques that make interaction with computer display as easy as pointing to real world objects is explored. Studies were conducted to investigate the way human point at objects naturally with their hand and to examine the inadequacy in existing pointing systems. Models that underpin the pointing strategy used in many of the previous interactive systems were formalized. A proof-of-concept prototype is built and evaluated from various user studies. Results from this thesis suggested that it is possible to allow natural user interaction with large displays using low-cost monocular computer vision. Furthermore, models developed and lessons learnt in this research can assist designers to develop more accurate and natural interactive systems that make use of human’s natural pointing behaviours
Addressing the problem of Interaction in fully immersive Virtual Environments: from raw sensor data to effective devices
Immersion into Virtual Reality is a perception of being physically present in a non-physical world. The perception is created by surrounding the user of the VR system with images, sound or other stimuli that provide an engrossing total environment. The use of technological devices such as stereoscopic cameras, head-mounted displays, tracking systems and haptic interfaces allows for user experiences providing a physical feeling of being in a realistic world, and the term “immersion” is a metaphoric use of the experience of submersion applied to representation, fiction or simulation.
One of the main peculiarity of fully immersive virtual reality is the enhancing of the simple passive viewing of a virtual environment with the ability to manipulate virtual objects inside it. This Thesis project investigates such interfaces and metaphors for the interaction and the manipulation tasks. In particular, the research activity conducted allowed the design of a thimble-like interface that can be used to recognize in real-time the human hand’s orientation and infer a simplified but effective model of the relative hand’s motion and gesture. Inside the virtual environment, users provided with the developed systems will be therefore able to operate with natural hand gestures in order to interact with the scene; for example, they could perform positioning task by moving, rotating and resizing existent objects, or create new ones from scratch.
This approach is particularly suitable when there is the need for the user to operate in a natural way, performing smooth and precise movements. Possible applications of the system to the industry are the immersive design in which the user can perform Computer- Aided Design (CAD) totally immersed in a virtual environment, and the operators training, in which the user can be trained on a 3D model in assembling or disassembling complex mechanical machineries, following predefined sequences.
The thesis has been organized around the following project plan:
- Collection of the relevant State Of The Art
- Evaluation of design choices and alternatives for the interaction hardware
- Development of the necessary embedded firmware
- Integration of the resulting devices in a complex interaction test-bed
- Development of demonstrative applications implementing the device
- Implementation of advanced haptic feedbac
Robots learn to behave: improving human-robot collaboration in flexible manufacturing applications
L'abstract è presente nell'allegato / the abstract is in the attachmen
Proceedings, MSVSCC 2013
Proceedings of the 7th Annual Modeling, Simulation & Visualization Student Capstone Conference held on April 11, 2013 at VMASC in Suffolk, Virginia
Human-Computer Interaction
In this book the reader will find a collection of 31 papers presenting different facets of Human Computer Interaction, the result of research projects and experiments as well as new approaches to design user interfaces. The book is organized according to the following main topics in a sequential order: new interaction paradigms, multimodality, usability studies on several interaction mechanisms, human factors, universal design and development methodologies and tools
Evaluating the use of augmented reality to facilitate assembly
Assembly is the process in which two or more objects are joined together through particular sequences and operations. Current practice utilises two-dimensional (2D) drawings as the main visualisation means to guide assembly. Other visualisation means such as three-dimensional (3D) manual and Virtual Reality (VR) technology have also been applied to assist in assembly. As an emerging technology, Augmented Reality (AR) integrates 3D images of virtual objects into a real-world workspace. The insertion of digitalised information into the real-world workspace using AR can provide workers with the means to implement correct assembly procedures with improved accuracy and reduced errors. Despite the substantial application of AR in assembly; related research has rarely been explored from a human cognitive perspective. The limited available cognitive research concerning the applications of AR visualisation means in assembly highlights the need for a structured methodology of addressing cognitive and useability issues for the application potentials of AR technology to be fully realised.This dissertation reviews the issues and discrepancies in using four types of visualisation means (2D drawings, 3D manual prints, VR, and AR) for guiding assembly, and investigates potential cognitive theories to underpin the benefits of animated AR in assembly. A theoretical framework is then put forward, which summarises existing mechanisms concerning visual-spatial information processing and THE Working Memory (WM) processing in the context of spatial cognition theory, active vision theory and THE WM theory, and raises the to-be-validated aspects of the above theories when transferring from the psychological arena to practical instances. Moreover, the dissertation formulates the methodology of configuring a prototype-animated AR system, and devising particular assembly tasks that are normally guided by reference to documentation and a test-bed with a series of experiments.Two experiments were conducted with three testing scenarios: experiment I concerns the evaluation in the first and second scenarios, while experiment II concerns the third scenario. In scenario 1, a small scale LEGO model was used as the assembly and experimental tester task to compare 3D manual prints and AR. This scenario measured the task performance and cognitive workload of using the system for assembly. The second scenario applied the knowledge gained from scenario 1 to the real construction piping assembly. Comparisons were then made as to productivity improvements, cost reduction and the reduction of rework between 2D isometric drawings and AR. Common findings from both scenarios revealed that the AR visualisation yielded shorter task completion time, less assembly errors and lower total task load. Evaluation from the real construction scenario also indicated that the animated AR visualisation significantly shortened the completion time (original time and rework time), payment to assemblers and cost on correcting erroneous assembly.Questionnaire feedback (including NASA task load index) (Hart 2006, 908) revealed that the animated AR visualisation better aided assembly comprehension, and better facilitated information retrieval and collaboration between human and guidance medium. Using the same LEGO tester task, the third scenario measured the training effects of using 3D manual prints and AR among novice assemblers. The results revealed that the learning curve of novice assemblers was reduced (faster learning) and task performance relevant to working memory was increased when implementing AR training. Useability evaluation was conducted based on classical useability methods, to assess the user interface regarding system improvements
Fluency and embodiment for robots acting with humans
Thesis (Ph. D.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2007.Includes bibliographical references (p. 225-234).This thesis is concerned with the notion of fluency in human-robot interaction (HRI), exploring cognitive mechanisms for robotic agents that would enable them to overcome the stop-and-go rigidity present in much of HRI to date. We define fluency as the ethereal yet manifest quality existent when two agents perform together at high level of coordination and adaptation, in particular when they are well-accustomed to the task and to each other. Based on mounting psychological and neurological evidence, we argue that one of the keys to this goal is the adaptation of an embodied approach to robot cognition. We show how central ideas from this psychological school are applicable to robot cognition and present a cognitive architecture making use of perceptual symbols, simulation, and perception-action networks. In addition, we demonstrate that anticipation of perceptual input, and in particular of the actions of others, are an important ingredient of fluent joint action. To that end, we show results from an experiment studying the effects of anticipatory action on fluency and teamwork, and use these results to suggest benchmark metrics for fluency. We also show the relationship between anticipatory action and a simulator approach to perception, through a comparative human subject study of an implemented cognitive architecture on the robot AUR, a robotic desk lamp, designed for this thesis. A result of this work is modeling the effect of practice on human-robot joint action, arguing that mechanisms that govern the passage of cognitive capabilities from a deliberate yet slower system to a faster, sub-intentional, and more rigid one, are crucial to fluent joint action in well-rehearsed ensembles. Theatrical acting theory serves as an inspiration for this work, as we argue that lessons from acting method can be applied to human-robot interaction.by Guy Hoffman.Ph.D
- …