Introducción a la háptica: nuevos dispositivos de entrada/salida

In this project, a review of the state of the art on virtual reality and augmented reality, input and output devices and haptic technology takes place in order to evaluate the role of haptic technology in the Human Computer Interaction (HCI). The starting point of the project is the broad increasing interest in technologies such as virtual reality or augmented reality, on one hand, and the haptic technology, on the other. We are seeing a great technological leap, mainly through audio visual dispositives (HMD Head- mounted display, headphones or glasses virtual reality) but with some forthcoming haptic devices, which are still unusual for the consumer. Nowadays, the technology seems to have reached enough maturity, in terms of hardware and software, for generating virtual environments of practical use by massive and unskilled user. Nevertheless, the desire sense of immersion is not yet complete: we can see, we can hear, but we can't touch. The requirements for these systems are becoming more and more demanding on realistic sensations to faithful recreate the immersion, by introducing the sense of touch as essential ingredient in the virtual environments. Then, the haptic technology comes as a natural step to all these audio visual novelties, as an essential role to build and manipulate virtual objects and scenes. Once the review of the state of the art is done, a conceptual design of a system based on input and output haptic is made to carry out a practical implementation of the solution and analyse the results obtained through some user tests.En el presente trabajo se realiza una revisión del estado del arte de realidad virtual y realidad aumentada , dispositivos de entrada y salida , y tecnología háptica con el objetiv o de ver de qué manera contribuye la háptica en la Interacción Persona - Computadora . El punto de partida del proyecto es el auge que están viviendo tecnologías como realidad virtual o realidad aumentada por un lado y la tecnología háptica por otro. Este au ge muestra el gran salto tenológico, en términos de madurez, entre los dispositives audivisuales (HMD, Head - mounted display , casco s o gafes de realidad virtual ) y los dispositives hápticos, aún de poco desarrollo para el usuario masivo. En la actualidad l a tecnología parece haber alcanzado la madurez suficiente, en términos de hardware y en software para la generación de entornos virtuales para el uso práctico por el público masivo y no especializado . Aun así, la sensación de inmersión que obtenemos no sie mpre resulta del todo completa ya que podemos ver, podemos oír, pero no podemos tocar. La necesidad actual de que estos sistemas sean cada vez más realistas obliga a introducir el sentido del tacto aumentando de forma considerable nuestra inmersión en ento rnos virtuales. Así pues, la tecnología háptica, se presenta como un paso natural a todas estas novedades audiovisuales, jugando un papel esencial a la hora de construir y manipular escenarios y objetos virtuales. Una vez realizada una revisión del est ado del arte, se realiza un diseño conceptual de un sistema de entrada y salida basado en háptica para llevar a cabo una implementación práctica de la solución y analizar los resultados obtenidos mediante la realización de ensayos y prueba

Realistic Interaction with Virtual Objects within Arm's Reach

The automotive industry requires realistic virtual reality applications more than other domains to increase the efficiency of product development. Currently, the visual quality of virtual invironments resembles reality, but interaction within these environments is usually far from what is known in everyday life. Several realistic research approaches exist, however they are still not all-encompassing enough to be usable in industrial processes. This thesis realizes lifelike direct multi-hand and multi-finger interaction with arbitrary objects, and proposes algorithmic and technical improvements that also approach lifelike usability. In addition, the thesis proposes methods to measure the effectiveness and usability of such interaction techniques as well as discusses different types of grasping feedback that support the user during interaction. Realistic and reliable interaction is reached through the combination of robust grasping heuristics and plausible pseudophysical object reactions. The easy-to-compute grasping rules use the objects’ surface normals, and mimic human grasping behavior. The novel concept of Normal Proxies increases grasping stability and diminishes challenges induced by adverse normals. The intricate act of picking-up thin and tiny objects remains challenging for some users. These cases are further supported by the consideration of finger pinches, which are measured with a specialized finger tracking device. With regard to typical object constraints, realistic object motion is geometrically calculated as a plausible reaction on user input. The resulting direct finger-based interaction technique enables realistic and intuitive manipulation of arbitrary objects. The thesis proposes two methods that prove and compare effectiveness and usability. An expert review indicates that experienced users quickly familiarize themselves with the technique. A quantitative and qualitative user study shows that direct finger-based interaction is preferred over indirect interaction in the context of functional car assessments. While controller-based interaction is more robust, the direct finger-based interaction provides greater realism, and becomes nearly as reliable when the pinch-sensitive mechanism is used. At present, the haptic channel is not used in industrial virtual reality applications. That is why it can be used for grasping feedback which improves the users’ understanding of the grasping situation. This thesis realizes a novel pressure-based tactile feedback at the fingertips. As an alternative, vibro-tactile feedback at the same location is realized as well as visual feedback by the coloring of grasp-involved finger segments. The feedback approaches are also compared within the user study, which reveals that grasping feedback is a requirement to judge grasp status and that tactile feedback improves interaction independent of the used display system. The considerably stronger vibrational tactile feedback can quickly become annoying during interaction. The interaction improvements and hardware enhancements make it possible to interact with virtual objects in a realistic and reliable manner. By addressing realism and reliability, this thesis paves the way for the virtual evaluation of human-object interaction, which is necessary for a broader application of virtual environments in the automotive industry and other domains.Stärker als andere Branchen benötigt die Automobilindustrie realistische Virtual Reality Anwendungen für eine effiziente Produktentwicklung. Während sich die visuelle Qualität virtueller Darstellungen bereits der Realität angenähert hat, ist die Interaktion mit virtuellen Umgebungen noch weit vom täglichen Erleben der Menschen entfernt. Einige Forschungsansätze haben sich mit realistischer Interaktion befasst, gehen aber nicht weit genug, um in industriellen Prozessen eingesetzt zu werden. Diese Arbeit realisiert eine lebensnahe mehrhändige und fingerbasierte Interaktion mit beliebigen Objekten. Dabei ermöglichen algorithmische und technische Verbesserungen eine realitätsnahe Usability. Außerdem werden Methoden für die Evaluation dieser Interaktionstechnik vorgestellt und benutzerunterstützende Greiffeedbackarten diskutiert. Die verlässliche und gleichzeitig realistische Interaktion wird durch die Kombination von robusten Greifheuristiken und pseudophysikalischen Objektreaktionen erreicht. Die das menschliche Greifverhalten nachbildenden Greifregeln basieren auf den Oberflächennormalen der Objekte. Die Reduktion negativer Einflüsse verfälschter Normalen und eine höhere Griffstabilität werden durch das neuartige Konzept der Normal Proxies erreicht. Dennoch bleibt für manche Nutzer das Aufnehmen von dünnen und kleinen Objekten problematisch. Diese Fälle werden zusätzlich durch die Einbeziehung von Fingerberührungen unterstützt, die mit einem speziellen Fingertracking Gerät erfasst werden. Plausible Objektreaktionen auf Benutzereingaben werden unter Berücksichtigung typischer Objekteinschränkungen geometrisch berechnet. Die Arbeit schlägt zwei Methoden zur Evaluierung der fingerbasierten Interaktion vor. Ein Expertenreview zeigt, dass sich erfahrene Benutzer sehr schnell in die Technik einfinden. In einer Benutzerstudie wird nachgewiesen, dass fingerbasierte Interaktion im hier untersuchten Kontext vor indirekter Interaktion mit einem Eingabegerät bevorzugt wird. Während letztere robuster zu handhaben ist, stellt die fingerbasierte Interaktion einen deutlich höheren Realismus bereit und erreicht mit den vorgeschlagenen Verbesserungen eine vergleichbare Verlässlichkeit. Um Greifsituationen transparent zu gestalten, realisiert diese Arbeit ein neuartiges druckbasiertes taktiles Feedback an den Fingerspitzen. Alternativ wird ein vibrotaktiles Feedback am gleichen Ort realisiert und visuelles Feedback durch die Einfärbung der griffbeteiligten Fingersegmente umgesetzt. Die verschiedenen Feedbackansätze werden in der Benutzerstudie verglichen. Dabei wird Greiffeedback als Voraussetzung identifiziert, um den Greifzustand zu beurteilen. Taktiles Feedback verbessert dabei die Interaktion unabhängig vom eingesetzten Display. Das merklich stärkere Vibrationsfeedback kann während der Interaktion störend wirken. Die vorgestellten Interaktionsverbesserungen und Hardwareerweiterungen ermöglichen es, mit virtuellen Objekten auf realistische und zuverlässige Art zu interagieren. Indem die Arbeit Realismus und Verlässlichkeit gleichzeitig adressiert, bereitet sie den Boden für die virtuelle Untersuchung von Mensch-Objekt Interaktionen und ermöglicht so einen breiteren Einsatz virtueller Techniken in der Automobilindustrie und in anderen Bereichen

Realistic Interaction with Virtual Objects within Arm's Reach

The automotive industry requires realistic virtual reality applications more than other domains to increase the efficiency of product development. Currently, the visual quality of virtual invironments resembles reality, but interaction within these environments is usually far from what is known in everyday life. Several realistic research approaches exist, however they are still not all-encompassing enough to be usable in industrial processes. This thesis realizes lifelike direct multi-hand and multi-finger interaction with arbitrary objects, and proposes algorithmic and technical improvements that also approach lifelike usability. In addition, the thesis proposes methods to measure the effectiveness and usability of such interaction techniques as well as discusses different types of grasping feedback that support the user during interaction. Realistic and reliable interaction is reached through the combination of robust grasping heuristics and plausible pseudophysical object reactions. The easy-to-compute grasping rules use the objects’ surface normals, and mimic human grasping behavior. The novel concept of Normal Proxies increases grasping stability and diminishes challenges induced by adverse normals. The intricate act of picking-up thin and tiny objects remains challenging for some users. These cases are further supported by the consideration of finger pinches, which are measured with a specialized finger tracking device. With regard to typical object constraints, realistic object motion is geometrically calculated as a plausible reaction on user input. The resulting direct finger-based interaction technique enables realistic and intuitive manipulation of arbitrary objects. The thesis proposes two methods that prove and compare effectiveness and usability. An expert review indicates that experienced users quickly familiarize themselves with the technique. A quantitative and qualitative user study shows that direct finger-based interaction is preferred over indirect interaction in the context of functional car assessments. While controller-based interaction is more robust, the direct finger-based interaction provides greater realism, and becomes nearly as reliable when the pinch-sensitive mechanism is used. At present, the haptic channel is not used in industrial virtual reality applications. That is why it can be used for grasping feedback which improves the users’ understanding of the grasping situation. This thesis realizes a novel pressure-based tactile feedback at the fingertips. As an alternative, vibro-tactile feedback at the same location is realized as well as visual feedback by the coloring of grasp-involved finger segments. The feedback approaches are also compared within the user study, which reveals that grasping feedback is a requirement to judge grasp status and that tactile feedback improves interaction independent of the used display system. The considerably stronger vibrational tactile feedback can quickly become annoying during interaction. The interaction improvements and hardware enhancements make it possible to interact with virtual objects in a realistic and reliable manner. By addressing realism and reliability, this thesis paves the way for the virtual evaluation of human-object interaction, which is necessary for a broader application of virtual environments in the automotive industry and other domains.Stärker als andere Branchen benötigt die Automobilindustrie realistische Virtual Reality Anwendungen für eine effiziente Produktentwicklung. Während sich die visuelle Qualität virtueller Darstellungen bereits der Realität angenähert hat, ist die Interaktion mit virtuellen Umgebungen noch weit vom täglichen Erleben der Menschen entfernt. Einige Forschungsansätze haben sich mit realistischer Interaktion befasst, gehen aber nicht weit genug, um in industriellen Prozessen eingesetzt zu werden. Diese Arbeit realisiert eine lebensnahe mehrhändige und fingerbasierte Interaktion mit beliebigen Objekten. Dabei ermöglichen algorithmische und technische Verbesserungen eine realitätsnahe Usability. Außerdem werden Methoden für die Evaluation dieser Interaktionstechnik vorgestellt und benutzerunterstützende Greiffeedbackarten diskutiert. Die verlässliche und gleichzeitig realistische Interaktion wird durch die Kombination von robusten Greifheuristiken und pseudophysikalischen Objektreaktionen erreicht. Die das menschliche Greifverhalten nachbildenden Greifregeln basieren auf den Oberflächennormalen der Objekte. Die Reduktion negativer Einflüsse verfälschter Normalen und eine höhere Griffstabilität werden durch das neuartige Konzept der Normal Proxies erreicht. Dennoch bleibt für manche Nutzer das Aufnehmen von dünnen und kleinen Objekten problematisch. Diese Fälle werden zusätzlich durch die Einbeziehung von Fingerberührungen unterstützt, die mit einem speziellen Fingertracking Gerät erfasst werden. Plausible Objektreaktionen auf Benutzereingaben werden unter Berücksichtigung typischer Objekteinschränkungen geometrisch berechnet. Die Arbeit schlägt zwei Methoden zur Evaluierung der fingerbasierten Interaktion vor. Ein Expertenreview zeigt, dass sich erfahrene Benutzer sehr schnell in die Technik einfinden. In einer Benutzerstudie wird nachgewiesen, dass fingerbasierte Interaktion im hier untersuchten Kontext vor indirekter Interaktion mit einem Eingabegerät bevorzugt wird. Während letztere robuster zu handhaben ist, stellt die fingerbasierte Interaktion einen deutlich höheren Realismus bereit und erreicht mit den vorgeschlagenen Verbesserungen eine vergleichbare Verlässlichkeit. Um Greifsituationen transparent zu gestalten, realisiert diese Arbeit ein neuartiges druckbasiertes taktiles Feedback an den Fingerspitzen. Alternativ wird ein vibrotaktiles Feedback am gleichen Ort realisiert und visuelles Feedback durch die Einfärbung der griffbeteiligten Fingersegmente umgesetzt. Die verschiedenen Feedbackansätze werden in der Benutzerstudie verglichen. Dabei wird Greiffeedback als Voraussetzung identifiziert, um den Greifzustand zu beurteilen. Taktiles Feedback verbessert dabei die Interaktion unabhängig vom eingesetzten Display. Das merklich stärkere Vibrationsfeedback kann während der Interaktion störend wirken. Die vorgestellten Interaktionsverbesserungen und Hardwareerweiterungen ermöglichen es, mit virtuellen Objekten auf realistische und zuverlässige Art zu interagieren. Indem die Arbeit Realismus und Verlässlichkeit gleichzeitig adressiert, bereitet sie den Boden für die virtuelle Untersuchung von Mensch-Objekt Interaktionen und ermöglicht so einen breiteren Einsatz virtueller Techniken in der Automobilindustrie und in anderen Bereichen

Multimodal feedback cues on manual lifting in virtual environments

Improper manipulation of real-world objects increases the risk of developing work- related back injuries. In an effort to reduce such a risk and encourage appropriate lifting and moving methods, a Virtual Environment (VE) was employed. Virtual simulations can be used for ergonomic analysis. In this work, the VEs made use of multiple feedback techniques to allow a person to estimate the forces acting on their lower back. A person's head and hand movements were tracked in real-time whilst manipulating an object. A NIOSH lifting equation was used to calculate and determine the Lifting Index whereby the results were conveyed in real time. Visual display feedback techniques were designed and the effect of cues to enhance user performance was experimentally evaluated. The feedback cues provide the user with information about the forces acting on their lower back as they perform manual lifting tasks in VEs. Four different methods were compared and contrasted: No Feedback, Text, Colour and Combined Colour and Text. This work also investigated various types of auditory feedback technique to support object manipulation in VEs. Auditory feedback has been demonstrated to convey information in computer applications effectively, but little work has been reported on the efficacy of such techniques, particularly for ergonomic design. Four different methods were compared and contrasted: No Feedback, White-noise, Pitch and Tempo. A combined Audio-Visual (AV) technique was also examined by mixing both senses. The effect of Tactile Augmentation was also examined. Three different weights (real) were used and the results obtained by experiment were compared with the experiment using virtual weights in order to evaluate whether or not the presence of a real weighted object enhanced people's sense of realism. The goals of this study were to explore various senses of feedback technique (visual, auditory and tactile), compare the performance characteristics of each technique and understand their relative advantages and drawbacks.EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Multimodal feedback cues on manual lifting in virtual environments

Improper manipulation of real-world objects increases the risk of developing work- related back injuries. In an effort to reduce such a risk and encourage appropriate lifting and moving methods, a Virtual Environment (VE) was employed. Virtual simulations can be used for ergonomic analysis. In this work, the VEs made use of multiple feedback techniques to allow a person to estimate the forces acting on their lower back. A person's head and hand movements were tracked in real-time whilst manipulating an object. A NIOSH lifting equation was used to calculate and determine the Lifting Index whereby the results were conveyed in real time. Visual display feedback techniques were designed and the effect of cues to enhance user performance was experimentally evaluated. The feedback cues provide the user with information about the forces acting on their lower back as they perform manual lifting tasks in VEs. Four different methods were compared and contrasted: No Feedback, Text, Colour and Combined Colour and Text. This work also investigated various types of auditory feedback technique to support object manipulation in VEs. Auditory feedback has been demonstrated to convey information in computer applications effectively, but little work has been reported on the efficacy of such techniques, particularly for ergonomic design. Four different methods were compared and contrasted: No Feedback, White-noise, Pitch and Tempo. A combined Audio-Visual (AV) technique was also examined by mixing both senses. The effect of Tactile Augmentation was also examined. Three different weights (real) were used and the results obtained by experiment were compared with the experiment using virtual weights in order to evaluate whether or not the presence of a real weighted object enhanced people's sense of realism. The goals of this study were to explore various senses of feedback technique (visual, auditory and tactile), compare the performance characteristics of each technique and understand their relative advantages and drawbacks

The feasibility of using virtual prototyping technologies for product evaluation

With the continuous development in computer and communications technology the use of computer aided design in design processes is becoming more commonplace. A wide range of virtual prototyping technologies are currently in development, some of which are commercially viable for use within a product design process. These virtual prototyping technologies range from graphics tablets to haptic devices. With the compression of design cycles the feasibility of using these technologies for product evaluation is becoming an ever more important consideration. This thesis begins by presenting the findings of a comprehensive literature review defining product design with a focus on product evaluation and a discussion of current virtual prototyping technologies. From the literature review it was clear that user involvement in the product evaluation process is critical. The literature review was followed by a series of interconnected studies starting with an investigation into design consultancies' access and use of prototyping technologies and their evaluation methods. Although design consultancies are already using photo-realistic renderings, animations and sometimes 3600 view CAD models for their virtual product evaluations, current virtual prototyping hardware and software is often unsatisfactory for their needs. Some emergent technologies such as haptic interfaces are currently not commonly used in industry. This study was followed by an investigation into users' psychological acceptance and physiological discomfort when using a variety of virtual prototyping tools for product evaluation compared with using physical prototypes, ranging from on-screen photo-realistic renderings to 3D 3600 view models developed using a range of design software. The third study then went on to explore the feasibility of using these virtual prototyping tools and the effect on product preference when compared to using physical prototypes. The forth study looked at the designer's requirements for current and future virtual prototyping tools, design tools and evaluation methods. In the final chapters of the thesis the relative strengths and weaknesses of these technologies were re-evaluated and a definitive set of user requirements based on the documentary evidence of the previous studies was produced. This was followed by the development of a speculative series of scenarios for the next generation of virtual prototyping technologies ranging from improvements to existing technologies through to blue sky concepts. These scenarios were then evaluated by designers and consumers to produce documentary evidence and recommendations for preferred and suitable combinations of virtual prototyping technologies. Such hardware and software will require a user interface that is intuitive, simple, easy to use and suitable for both the designers who create the virtual prototypes and the consumers who evaluate them

Interactive Fiction in Cinematic Virtual Reality: Epistemology, Creation and Evaluation

This dissertation presents the Interactive Fiction in Cinematic Virtual Reality (IFcVR), an interactive digital narrative (IDN) that brings together the cinematic virtual reality (cVR) and the creation of virtual environments through 360\ub0 video within an interactive fiction (IF) structure. This work is structured in three components: an epistemological approach to this kind of narrative and media hybrid; the creation process of IFcVR, from development to postproduction; and user evaluation of IFcVR. In order to set the foundations for the creation of interactive VR fiction films, I dissect the IFcVR by investigating the aesthetics, narratological and interactive notions that converge and diverge in it, proposing a medium-conscious narratology for this kind of artefact. This analysis led to the production of an IFcVR functional prototype: \u201cZENA\u201d, the first interactive VR film shot in Genoa. ZENA\u2019s creation process is reported proposing some guidelines for interactive and immersive film-makers. In order to evaluate the effectiveness of the IFcVR as an entertaining narrative form and a vehicle for diverse types of messages, this study also proposes a methodology to measure User Experience (UX) on IFcVR. The full evaluation protocol gathers both qualitative and quantitative data through ad hoc instruments. The proposed protocol is illustrated through its pilot application on ZENA. Findings show interactors' positive acceptance of IFcVR as an entertaining experience

Structured evaluation of training in virtual environments

Virtual Environments (VEs) created through Virtual Reality (VR) technologies have been suggested as potentially beneficial for a number of applications. However a review of VEs and VR has highlighted the main barriers to implementation as: current technological limitations; usability issues with various systems; a lack of real applications; and therefore little proven value of use. These barriers suggest that industry would benefit from some structured guidance for developing effective VEs. To examine this ‘training’ was chosen to be explored, as it has been suggested as a potential early use of VEs and is of importance to many sectors. A review of existing case studies on VE training applications (VETs) examined type of training applications and VR systems being considered; state of development of these applications and results of any evaluation studies. In light of these case studies, it was possible to focus this work on the structured evaluation of training psycho-motor skills using VEs created by desktop VR. In order to perform structured evaluation, existing theories of training and evaluation were also reviewed. Using these theories, a framework for developing VETs was suggested. Applying this framework, two VETs were proposed, specified, developed and evaluated. Conclusions of this work highlighted the many areas in the development process of an effective VET that still need addressing. In particular, in the proposal stage, it is necessary to provide some guidance on the appropriateness of VET for particular tasks. In the specification and building stages, standard formats and techniques are required in order to guide the VE developer(s) in producing an effective VET. Finally in the evaluation stage, there are still tools required that highlight the benefits of VET and many more evaluation studies needed to contribute information back to the development process. Therefore VEs are still in their early stages and this work unifies existing work in the area specifically on training and highlights the gaps that need to be addressed before widespread implementation

Virtual Environments. Seminar - Sommersemester 2003

Dieser Bericht stellt die Ergebnisse des Seminars Virtual Environments (VE) zusammen. Ein wichtiges Ziel von VE ist die Immersion, die Einbindung des Benutzers als aktiven Teilnehmer in eine computergenerierte Welt. Voraussetzung dafür sind Techniken zur Simulation von Lebendigen virtuellen Welten, also zur Simulation von 3D-Szenen mit realistischem Verhalten. Es geht dabei um Kollisionserkennungsalgorithmen, haptisches Rendering, Navigations- und Interaktionstechniken, programmierbare Grafik-Hardware, verteilte virtuelle Welten bis hin zur Modellierung und Simulation von virtuellen Menschen. Die virtuelle Realität hat sich inzwischen in verschiedenen Anwendungsbereichen durchgesetzt und wird auch im Rahmen des SFB 588 Humanoide Roboter - Lernende und kooperierende multimodale Roboter für die Simulation des humanoiden Roboters und die Evaluierung der Mensch-Roboter-Schnittstelle eingesetzt

Machine Learning in Sensors and Imaging

Machine learning is extending its applications in various fields, such as image processing, the Internet of Things, user interface, big data, manufacturing, management, etc. As data are required to build machine learning networks, sensors are one of the most important technologies. In addition, machine learning networks can contribute to the improvement in sensor performance and the creation of new sensor applications. This Special Issue addresses all types of machine learning applications related to sensors and imaging. It covers computer vision-based control, activity recognition, fuzzy label classification, failure classification, motor temperature estimation, the camera calibration of intelligent vehicles, error detection, color prior model, compressive sensing, wildfire risk assessment, shelf auditing, forest-growing stem volume estimation, road management, image denoising, and touchscreens