Head-Tracking Haptic Computer Interface for the Blind

In today’s heavily technology-dependent society, blind and visually impaired people are becoming increasingly disadvantaged in terms of access to media, information, electronic commerce, communications and social networks. Not only are computers becoming more widely used in general, but their dependence on visual output is increasing, extending the technology further out of reach for those without sight. For example, blindness was less of an obstacle for programmers when command-line interfaces were more commonplace, but with the introduction of Graphical User Interfaces (GUIs) for both development and final applications, many blind programmers were made redundant (Alexander, 1998; Siegfried et al., 2004). Not only are images, video and animation heavily entrenched in today’s interfaces, but the visual layout of the interfaces themselves hold important information which is inaccessible to sightless users with existing accessibility technology

Substitution visuelle par électro-stimulation linguale : étude des procédés affectant la qualité de perception

Our perception of the world is by essence multimodal. Various sensory cues (vision, hearing, touch, etc.) are hence integrated to allow us the execution of daily life activities. When one sense organ is missing or impaired, in some circumstances, the central nervous system can replace or reinforce the missing sensory information by using reliable sensory information coming from an other intact sense organ. This doctoral thesis was designed to explore this so-called ‘sensory substitution' concept through the exploitation of a visual-to-tactile sensory substitution device. the Tongue Display Unit (TDU) is made of a 12x12 electro-tactile matrix capable of rendering visual scences on the dorsal part of the tongue.However, the effectiveness of this device suffers from numerous drawbacks among which two main problems : (i) the problem of ‘fading' (mitigation) which corresponds to a rapid loss of perception of electrostimulation signal by the individuals and (ii) the problem of multimodal fusion. In the present work, we propose to investigate these problems and their effects on the quality of perception, by (i) applying some eye-inspired mechanisms (micro-saccades) to the electro-tactile signal on the tongue, (ii) studying how individuals perceive a picture signal with multi-modal perceptual (visual stimulus + substitution on the tongue) and (iii) the effect produced contradictions of information in this context.This work evidence the importance of applying saccades and sparkle mechanisms in electro-tactile signal to resolve the problem of fading especially when stimulation periods are relatively long. Furthermore, the experiments we have conducted on multi-modal perception and contradictions suggested that the perception of image information through the TDU was comparable to that of vision. Finally, we were able to identify a conflicting information fusion effect between visual perception and electrotactile tongue perception. This phenomenon is known as Mc Gurck effect (fusion of visual and auditory information). Understanding the effect of these processes to improve the quality of perception through the visual substitution paradigm and the importance of multi-modal fusion effects during this thesis could be applied in various areas where these effects are present or required, especially in the emerging field of prosthetic vision.Notre perception du monde est multimodale. Les différents sens (vision, audition, tactile, etc) sont coordonnés entre eux afin d'accomplir nos tâches quotidiennes. Dans le cas où un organe des sens connaît un dysfonctionnement, le système nerveux central est capable de remplacer ou de renforcer l'information manquante en faisant appel à d'autres modalités sensorielles intactes. Le cadre conceptuel de ce remplacement d'un sens par un autre est le paradigme de substitution sensorielle. Cette thèse se situe dans ce contexte et, pour explorer certains aspects de ce paradigme, exploite un dispositif d'électro-stimulation linguale, le Tongue Display Unit (TDU), une matrice de 12x12 électrodes destinée à restituer sous la forme de voltages variables des scènes visuelles sur la langue.L'efficacité de ce dispositif souffre de nombreux problèmes au rang desquels le problème de fading (atténuation) qui correspond à une perte rapide de perception du signal d'électro-stimulation par les sujets et celui de la fusion de perceptions multi-modales qui, nous le verrons, se complique avec l'usage du TDU. Dans cette thèse, nous proposons d'étudier l'effet sur la qualité de perception et sur l'atténuation du signal de l'adjonction au signal image de saccades et de tremblements, processus qui nous ont été inspirés par la vision. Nous étudions également la façon dont les sujets perçoivent un signal image en cas de multi-modalité perceptive (vision + substitution visuelle sur la langue) et l'effet que produisent des contradictions d'information dans ce contexte.Ce travail met en évidence l'importance de la présence de saccades ou de scintillement dans le signal d'électro-stimulation pour contrer les effets de fading : grâce à ces procédés, même lorsque les stimulations sont longues, les sujets continuent de percevoir correctement le signal image. De plus, les expériences de perception multi-modale et de contradictions nous ont permis de montrer que la perception d'informations de type image par l'intermédiaire du TDU était comparable à celle de la vision. Enfin, nous avons pu mettre en évidence un effet de fusion d'informations contradictoires entre la perception visuelle et linguale, connu dans le cadre de la fusion d'information visuo-auditives sous le nom d'effet Mc Gurck.La compréhension de l'effet de procédés destinés à améliorer la perception dans le cadre de la substitution visuelle et celle de l'importance des effets de fusion multi-modale dégagée au cours de thèse devraient trouver application dans de nombreux domaines où ces effets sont présents ou requis, en particulier dans le domaine émergent de la vision prothétique

Multimodal interaction: developing an interaction concept for a touchscreen incorporating tactile feedback

The touchscreen, as an alternative user interface for applications that normally require mice and keyboards, has become more and more commonplace, showing up on mobile devices, on vending machines, on ATMs and in the control panels of machines in industry, where conventional input devices cannot provide intuitive, rapid and accurate user interaction with the content of the display. The exponential growth in processing power on the PC, together with advances in understanding human communication channels, has had a significant effect on the design of usable, human-factored interfaces on touchscreens, and on the number and complexity of applications available on touchscreens. Although computer-driven touchscreen interfaces provide programmable and dynamic displays, the absence of the expected tactile cues on the hard and static surfaces of conventional touchscreens is challenging interface design and touchscreen usability, in particular for distracting, low-visibility environments. Current technology allows the human tactile modality to be used in touchscreens. While the visual channel converts graphics and text unidirectionally from the computer to the end user, tactile communication features a bidirectional information flow to and from the user as the user perceives and acts on the environment and the system responds to changing contextual information. Tactile sensations such as detents and pulses provide users with cues that make selecting and controlling a more intuitive process. Tactile features can compensate for deficiencies in some of the human senses, especially in tasks which carry a heavy visual or auditory burden. In this study, an interaction concept for tactile touchscreens is developed with a view to employing the key characteristics of the human sense of touch effectively and efficiently, especially in distracting environments where vision is impaired and hearing is overloaded. As a first step toward improving the usability of touchscreens through the integration of tactile effects, different mechanical solutions for producing motion in tactile touchscreens are investigated, to provide a basis for selecting suitable vibration directions when designing tactile displays. Building on these results, design know-how regarding tactile feedback patterns is further developed to enable dynamic simulation of UI controls, in order to give users a sense of perceiving real controls on a highly natural touch interface. To study the value of adding tactile properties to touchscreens, haptically enhanced UI controls are then further investigated with the aim of mapping haptic signals to different usage scenarios to perform primary and secondary tasks with touchscreens. The findings of the study are intended for consideration and discussion as a guide to further development of tactile stimuli, haptically enhanced user interfaces and touchscreen applications

Multi-Finger Haptic Devices Integrating Miniature Short-Stroke Actuators

The omnipresence of electronic devices in our everyday life goes together with a trend that makes us always more immersed during their utilization. By immersion, we mean that during the development of a new product, it is more and more required to stimulate several senses of the user so as to make the product more attractive. The sense of touch does not escape the rule and is more and more considered. Definitely democratized by its integration in smart phones with touchscreens, the haptic feedback allows enhancing the human-machine interactions in many ways. For instance by improving the comfort of use of a button through the modification of its force feedback. It can also offer an interactive experience during the manipulation of digital information and even improve the communication, particularly through the internet and for blind people, with the introduction of non-verbal signals. For these reasons, the present thesis focuses on the conception of multi-finger haptic devices, a new kind of peripherals integrating multiple actuators and capable of providing a fully programmable force feedback to the user's fingers. A global methodology is presented, outlining the different constituents necessary for their conception: actuator, sensor, control, communication and software user interface. Then, generic tools corresponding to the two first elements are presented. An accurate modeling of miniature electromagnetic short-stroke actuators is made possible thanks to the combination of 3D finite element modeling (FEM) and design of experiments (DOE). The non-usual behavior of magnetic flux lines in miniature actuators with relatively large airgaps imposes to avoid simplified analytical models and to use the reliable results of finite elements. The long computation times required by 3D FEM are balanced by the use of selective DOE making the modeling methodology easily adaptable, rapid and accurate. The parametrical model of the force provided by the modeling methodology is then integrated in a full parametrical setup allowing for the optimization of the actuator force using a conventional algorithm. The advantage of the parametrical optimization is that complementary non-linear constraints such as weight and temperature can be added, making the model multi-physic. Then, several original position measurement techniques using existing sensors are developed including a low-cost custom single-photointerrupter sensor allowing for direction discrimination for fast-prototyping and a hybrid sensing method using tiny Hall sensors and taking advantage of the leaks of the main actuator magnet. Two innovative self-sensing methods are then presented, allowing for the measurement of the mover position of linear short-stroke actuators. The first solution estimates the position of the coil by measuring the acceleration through the back emf. However in this case, a constant acceleration is required, which strongly restrains the application scope. The second solution allows for a real-time measurement of the position thanks to a passive oscillating RLC circuit influenced by the variation of the coil impedance. All the solutions presented are low-cost, compact and require few computation resources. Finally, in order to illustrate the methodology proposed along the thesis, several prototypes are fabricated, giving an overview of the possibilities offered by multi-finger haptic devices. A haptic numeric pad is notably used in an experiment made in collaboration with the University Service of Child and Adolescent Psychiatry in Lausanne with the aim of improving the impaired emotional processing of psychotic adolescents. Moreover, the successful identification of several touch sensations on the same haptic pad lays the first stones of a new tactile language