Standardized evaluation of haptic rendering systems

The development and evaluation of haptic rendering algorithms presents two unique challenges. Firstly, the haptic information channel is fundamentally bidirectional, so the output of a haptic environment is fundamentally dependent on user input, which is difficult to reliably reproduce. Additionally, it is difficult to compare haptic results to real-world, "gold standard" results, since such a comparison requires applying identical inputs to real and virtual objects and measuring the resulting forces, which requires hardware that is not widely available. We have addressed these challenges by building and releasing several sets of position and force information, collected by physically scanning a set of real-world objects, along with virtual models of those objects. We demonstrate novel applications of this data set for the development, debugging, optimization, evaluation, and comparison of haptic rendering algorithms

Evaluating the Influence of Haptic Force-Feedback on 3D Selection Tasks using Natural Egocentric Gestures

Immersive Virtual Environments (IVEs) allow participants to interact with their 3D surroundings using natural hand gestures. Previous work shows that the addition of haptic feedback cues improves performance on certain 3D tasks. However, we believe this is not true for all situations. Depending on the difficulty of the task, we suggest that we should expect differences in the ballistic movement of our hands when presented with different types of haptic force-feedback conditions. We investigated how hard, soft and no haptic force-feedback responses, experienced when in contact with the surface of an object, affected user performance on a task involving selection of multiple targets. To do this, we implemented a natural egocentric selection interaction technique by integrating a two-handed large-scale force-feedback device in to a CAVE (TM)-like IVE system. With this, we performed a user study where we show that participants perform selection tasks best when interacting with targets that exert soft haptic force-feedback cues. For targets that have hard and no force-feedback properties, we highlight certain associated hand movement that participants make under these conditions, that we hypothesise reduce their performance

Doctor of Philosophy

dissertationVirtual environments provide a consistent and relatively inexpensive method of training individuals. They often include haptic feedback in the form of forces applied to a manipulandum or thimble to provide a more immersive and educational experience. However, the limited haptic feedback provided in these systems tends to be restrictive and frustrating to use. Providing tactile feedback in addition to this kinesthetic feedback can enhance the user's ability to manipulate and interact with virtual objects while providing a greater level of immersion. This dissertation advances the state-of-the-art by providing a better understanding of tactile feedback and advancing combined tactilekinesthetic systems. The tactile feedback described within this dissertation is provided by a finger-mounted device called the contact location display (CLD). Rather than displaying the entire contact surface, the device displays (feeds back) information only about the center of contact between the user's finger and a virtual surface. In prior work, the CLD used specialized two-dimensional environments to provide smooth tactile feedback. Using polygonal environments would greatly enhance the device's usefulness. However, the surface discontinuities created by the facets on these models are rendered through the CLD, regardless of traditional force shading algorithms. To address this issue, a haptic shading algorithm was developed to provide smooth tactile and kinesthetic interaction with general polygonal models. Two experiments were used to evaluate the shading algorithm. iv To better understand the design requirements of tactile devices, three separate experiments were run to evaluate the perception thresholds for cue localization, backlash, and system delay. These experiments establish quantitative design criteria for tactile devices. These results can serve as the maximum (i.e., most demanding) device specifications for tactile-kinesthetic haptic systems where the user experiences tactile feedback as a function of his/her limb motions. Lastly, a revision of the CLD was constructed and evaluated. By taking the newly evaluated design criteria into account, the CLD device became smaller and lighter weight, while providing a full two degree-of-freedom workspace that covers the bottom hemisphere of the finger. Two simple manipulation experiments were used to evaluate the new CLD device

INTERACTION BETWEEN SIGNAL COMPLEXITY AND PHYSICAL ACTIVITY IN VIBRO-TACTILE COMMUNICATION

Master'sMASTER OF ART

Development of an Inexpensive, Haptic Graphical Display Device

A finger-worn haptic device capable of rendering 2-D graphics through vibrotactile feedback is presented. The device development is presented from its initial stages of being a stylus design using a photo-interrupter optical sensor and pager-motor actuator to a small case worn on the finger using a RGB color sensor and a piezoelectric actuator. Testing of the latest prototype design shows that it has a spatial sensitivity (\u3c2mm) comparable to natural touch (~1mm) and can be used to output a variety of vibrotactile textures. The design can be expanded for a multiple finger, independent device, while remaining affordable (\u3c$100) and highly portable (\u3c500g)

The feasibility of using virtual prototyping technologies for product evaluation

With the continuous development in computer and communications technology the use of computer aided design in design processes is becoming more commonplace. A wide range of virtual prototyping technologies are currently in development, some of which are commercially viable for use within a product design process. These virtual prototyping technologies range from graphics tablets to haptic devices. With the compression of design cycles the feasibility of using these technologies for product evaluation is becoming an ever more important consideration. This thesis begins by presenting the findings of a comprehensive literature review defining product design with a focus on product evaluation and a discussion of current virtual prototyping technologies. From the literature review it was clear that user involvement in the product evaluation process is critical. The literature review was followed by a series of interconnected studies starting with an investigation into design consultancies' access and use of prototyping technologies and their evaluation methods. Although design consultancies are already using photo-realistic renderings, animations and sometimes 3600 view CAD models for their virtual product evaluations, current virtual prototyping hardware and software is often unsatisfactory for their needs. Some emergent technologies such as haptic interfaces are currently not commonly used in industry. This study was followed by an investigation into users' psychological acceptance and physiological discomfort when using a variety of virtual prototyping tools for product evaluation compared with using physical prototypes, ranging from on-screen photo-realistic renderings to 3D 3600 view models developed using a range of design software. The third study then went on to explore the feasibility of using these virtual prototyping tools and the effect on product preference when compared to using physical prototypes. The forth study looked at the designer's requirements for current and future virtual prototyping tools, design tools and evaluation methods. In the final chapters of the thesis the relative strengths and weaknesses of these technologies were re-evaluated and a definitive set of user requirements based on the documentary evidence of the previous studies was produced. This was followed by the development of a speculative series of scenarios for the next generation of virtual prototyping technologies ranging from improvements to existing technologies through to blue sky concepts. These scenarios were then evaluated by designers and consumers to produce documentary evidence and recommendations for preferred and suitable combinations of virtual prototyping technologies. Such hardware and software will require a user interface that is intuitive, simple, easy to use and suitable for both the designers who create the virtual prototypes and the consumers who evaluate them

Surface Geometry and the Haptic Rendering of Rigid Point Contacts

This thesis examines the haptic rendering of rigid point contacts in virtual simulations. The haptic renderers generate force feedback so that the operator can interact with the virtual scenes in a realistic way. They must be able to recreate the physical phenomena experienced in the real world without displaying any haptic artifacts. The existing renderers are decomposed into a projection function and a regulation scheme. It is shown that the pop-through artifact, whereby the virtual tool instantaneously jumps between two distant surface points, is caused whenever the operator encounters a singularity within the renderer's projection function. This was well known for the minimum distance based renderers, but it is shown here that such singularities arise with the constraint based renderers as well. A new projection function is designed to minimize the existence of singularities within the model. When paired with an appropriate regulation scheme, this forms the proposed mapping renderer. The new projection is calculated by mapping the model onto a canonical shape where the haptic problem is trivial, e.g. a circle in the case of a 2D model of genus zero, which avoids pop-through on smooth models. The haptic problem is then recast as a virtual constraint problem, where the traditional regulation schemes, designed originally for planar surfaces, are shown to introduce a velocity dependent error on curved surfaces that can distort the model's rendering and to couple the regulation towards and dynamics along the constraint. Set stabilization control, based on feedback linearizing the haptic device with respect to a virtual output consisting of coordinates transversal and tangential to the model surface, is proposed as an alternative. It is shown to be able to decouple the system into transversal and tangential subsystems that can then be made asymptotically stable and assigned arbitrary dynamics, respectively

Taktile Interaktion auf flächigen Brailledisplays

Für den Zugang zu grafischen Benutzungsoberflächen (GUIs) stehen blinden Menschen so genannte Screenreader und Braillezeilen zur Verfügung. Diese ermöglichen zwar das nicht-visuelle Wahrnehmen textueller Inhalte, allerdings kein effektives Arbeiten mit bildlichen Darstellungen. Neuartige taktile Flächendisplays können eine geeignete Lösung für den interaktiven Zugang zu tastbaren Grafiken darstellen und somit die Interaktionsmöglichkeiten blinder Benutzer im Umgang mit grafischen Anwendungen bereichern. Beispielsweise erlauben derartige Geräte nicht nur das Erkunden räumlicher Anordnungen, sondern darüber hinaus auch die kombinierte Ausgabe von Braille, Grafik und semi-grafischen Elementen. Um die deutlich größere Menge an gleichzeitig darstellbaren Informationen beherrschbar zu machen, sind neben entsprechenden Inhaltsaufbereitungen und Navigationsmechanismen auch geeignete Orientierungshilfen bereitzustellen. Im Rahmen der vorliegenden Arbeit wurde am Beispiel der BrailleDis Geräte der Metec AG, welche eine taktile Ausgabefläche von 120 mal 60 Stiften bereitstellen, untersucht, inwieweit flächige Brailledisplays blinden Menschen eine effektive und effiziente Bedienung grafischer Benutzungsoberflächen ermöglichen. Neben dem Zugang zur GUI selbst sowie dem Lesen von Texten stellt dabei insbesondere das Arbeiten mit Grafiken einen wichtigen Aspekt dar. Um die Bedienung auf einem taktilen Flächendisplay zu erleichtern, ist eine konsistente Organisation der Inhalte hilfreich. Hierfür wurde ein neuartiges taktiles Fenstersystem umgesetzt, welches die Ausgabe nicht nur in mehrere disjunkte Bereiche unterteilt, sondern auch verschiedene taktile Darstellungsarten unterstützt. Zur Systematisierung der Gestaltung und Evaluation derartiger taktiler Benutzungsoberflächen sowie der darin stattfindenden Benutzerinteraktionen wurde zunächst eine Taxonomie erarbeitet. Dabei wurden neben der Interaktion selber, welche durch die Ein-und Ausgabe sowie die Handbewegungen des Benutzers beschrieben werden kann, auch die Benutzerintention in Form von taktilen Elementaraufgaben sowie die technischen Spezifikationen des Geräts mit einbezogen. Basierend auf der Taxonomie wurden anschließend relevante Aspekte identifiziert, welche in mehreren Benutzerstudien mit insgesamt 46 blinden und hochgradig sehbehinderten Menschen untersucht wurden. Die betrachteten Untersuchungsfragen betrafen dabei einerseits die Effektivität der Ausgabe in Form verschiedener taktiler Ansichtsarten sowie die Eingabe und Erkundung durch den Benutzer, andererseits auch Aspekte zur Effizienz konkreter Interaktionstechniken. Als Ergebnis der einzelnen Studien wurden abschließend konkrete Empfehlungen zur Umsetzung von Benutzungsoberflächen auf flächigen Brailledisplays gegeben. Diese beinhalten insbesondere Aspekte zur Ergonomie von taktilen Flächendisplays, zur Anzeige von textuellen Inhalten, zur Darstellung und Interaktion mit grafischen Inhalten sowie zu Orientierungshilfen. Insgesamt konnte mit Hilfe der Benutzerstudien gezeigt werden, dass flächige Brailledisplays blinden Menschen einen effektiven und effizienten Zugang zu grafischen Benutzungsoberflächen ermöglichen. Verschiedene taktile Darstellungsarten können dabei das Lösen unterschiedlicher Aufgaben unterstützen. Generell erfordert die flächige Interaktion vom Benutzer allerdings auch die Erweiterung seiner konventionellen Erkundungs-und Eingabestrategien. Die Bereitstellung neuartiger Interaktionstechniken zur Unterstützung der Orientierung kann die Effizienz zusätzlich steigern.Blind people normally use screen readers as well as single-lined refreshable Braille displays for accessing graphical user interfaces (GUIs). These technologies allow for a non-visual perception of textual content but not for an effective handling of visual illustrations. Novel two-dimensional tactile pin-matrix devices are an appropriate solution to interactively access tactual graphics. In this way, they can enrich the interaction possibilities of blind users in dealing with graphical applications. For instance, such devices enable the exploration of spatial arrangements and also combine output of Braille, graphics and semi-graphical elements. To make the high amount of simultaneously presented information perceivable and efficiently usable for blind users, an adequate preparation of content as well as adapted navigation and orientation mechanisms must be provided. In this thesis the BrailleDis devices of Metec AG, which have a tactile output area of 120 times 60 pins, were used. The goal was to investigate to what extent large pin-matrix devices enable blind people to use graphical user interfaces effectively and efficiently. Access to the GUI itself, reading text, and dealing with graphics are the main aspects of the application area of such devices. To facilitate the operation on a two-dimensional pin-matrix device a consistent organization of the content is helpful. Therefore, a novel tactile windowing system was implemented which divides the output area into multiple disjunctive regions and supports diverse tactile information visualizations. Moreover, a taxonomy was developed to systematize the design and evaluation of tactile user interfaces. Apart from interaction that can be described by input and output as well as hand movements, the taxonomy includes user intention in terms of interactive task primitives and technical specifications of the device. Based on the taxonomy, relevant aspects of tactile interaction were identified. These aspects were examined in multiple user studies with a total of 46 blind and visually impaired participants. The following research topics were considered during the user studies: 1. the effectiveness of diverse tactile view types (output), 2. user input and exploration, and 3. the efficiency of specific interaction techniques. As a result, practical recommendations for implementing user interfaces on two-dimensional pin-matrix devices were given. These recommendations include ergonomic issues of physical devices as well as design considerations for textual and graphical content as well as orientation aids. In summary, the user studies showed that two-dimensional pin-matrix devices enable blind people an effective and efficient access to graphical user interfaces. Diverse tactile information visualizations can support users to fulfill various tasks. In general, two-dimensional interaction requires the extension of conventional exploration and input strategies of users. The provision of novel interaction techniques for supporting orientation can help to increase efficiency even more

Exploring the Influence of Haptic Force Feedback on 3D Selection

This thesis studies the effects of haptic force feedback on 3D interaction performance. To date, Human-Computer Interaction (HCI) in three dimensions is not well understood. Within platforms, such as Immersive Virtual Environments (IVEs), implementing `good' methods of interaction is difficult. As reflected by the lack of 3D IVE applications in common use, typical performance constraints include inaccurate tracking, lack of additional sensory inputs, in addition to general design issues related to the implemented interaction technique and connected input devices. In total, this represents a broad set of multi-disciplinary challenges. By implementing techniques that address these problems, we intend to use IVE platforms to study human 3D interaction and the effects of different types of feedback. A promising area of work is the development of haptic force feedback devices. Also called haptic interfaces, these devices can exert a desired force onto the user simulating a physical interaction. When described as a sensory cue, it is thought that this information is important for the selection and manipulation of 3D objects. To date, there are a lot of studies investigating how best to integrate haptic devices within IVEs. Whilst there are still fundamental integration and device level problems to solve, previous work demonstrates that haptic force feedback can improve 3D interaction performance. By investigating this claim further, this thesis explores the role of haptic force feedback on 3D interaction performance in more detail. In particular, we found additional complexities whereby different types of haptic force feedback conditions can either help but also hinder user performance. By discussing these new results, we begin to examine the utility of haptic force feedback. By focusing our user studies on 3D selection, we explored the influence of haptic force feedback on the strategies taken to target virtual objects when using either `distal' and `natural' interaction technique designs. We first outlined novel methods for integrating and calibrating large scale haptic devices within a CAVE-like IVE. Secondly, we described our implementation of distal and natural selection techniques tailored to the available hardware, including the collision detection mechanisms used to render different haptic responses. Thirdly, we discussed the evaluation framework used to assess different interaction techniques and haptic force feedback responses within a common IVE setup. Finally, we provide a detailed assessment of user performance highlighting the effects of haptic force feedback on 3D selection, which is the main contribution of this work. We expect the presented findings will add to the existing literature that evaluates novel 3D interaction technique designs for IVEs. We also hope that this thesis will provide a basis to develop future interaction models that include the effects of haptic force feedback