Making touch-based kiosks accessible to blind users through simple gestures

Touch-based interaction is becoming increasingly popular and is commonly used as the main interaction paradigm for self-service kiosks in public spaces. Touch-based interaction is known to be visually intensive, and current non-haptic touch-display technologies are often criticized as excluding blind users. This study set out to demonstrate that touch-based kiosks can be designed to include blind users without compromising the user experience for non-blind users. Most touch-based kiosks are based on absolute positioned virtual buttons which are difficult to locate without any tactile, audible or visual cues. However, simple stroke gestures rely on relative movements and the user does not need to hit a target at a specific location on the display. In this study, a touch-based train ticket sales kiosk based on simple stroke gestures was developed and tested on a panel of blind and visually impaired users, a panel of blindfolded non-visually impaired users and a control group of non-visually impaired users. The tests demonstrate that all the participants managed to discover, learn and use the touch-based self-service terminal and complete a ticket purchasing task. The majority of the participants completed the task in less than 4 min on the first attempt

Mobile assistive technologies for the visually impaired

There are around 285 million visually impaired people worldwide, and around 370,000 people are registered as blind or partially sighted in the UK. Ongoing advances in information technology (IT) are increasing the scope for IT-based mobile assistive technologies to facilitate the independence, safety, and improved quality of life of the visually impaired. Research is being directed at making mobile phones and other handheld devices accessible via our haptic (touch) and audio sensory channels. We review research and innovation within the field of mobile assistive technology for the visually impaired and, in so doing, highlight the need for successful collaboration between clinical expertise, computer science, and domain users to realize fully the potential benefits of such technologies. We initially reflect on research that has been conducted to make mobile phones more accessible to people with vision loss. We then discuss innovative assistive applications designed for the visually impaired that are either delivered via mainstream devices and can be used while in motion (e.g., mobile phones) or are embedded within an environment that may be in motion (e.g., public transport) or within which the user may be in motion (e.g., smart homes)

A review of the current trends and future directions of camera barcode reading

Modern mobile phones or smartphones have become a pervasive and affordable device for users at different levels of age around the world. Smartphones equipped with many useful sensors, including camera, barometer, accelerometer, and digital compass. The sensors on smartphones attracted researchers and developers to develop mobile applications (apps) and study the potential use of the sensors to support daily life activities. Unlike other types of sensor, the smartphone camera has been underutilized. Analysis of the literature suggested that smartphone camera mainly serves for personal and social photography. Practically, a smartphone camera can be used as an imaging device for reading a barcode. Although barcode has been used for identifying products and items, the use of a smartphone camera as a reading device has not been explored thoroughly. Further, scholarly resources describing the fundamental knowledge of smartphone camera barcode reading is not available in the literature which could be the reason contributed to slow research progress of the domain. Therefore, this study aims to review the current trends and future directions of smartphone camera for barcode reading. Specifically, the study reviews the literature on the types of applications that are currently available and run on the standard mobile platform for reading a barcode. It also analyzes the necessary components that made up barcode reading apps. Further, the review identifies technical and non-technical issues that are critical for the development of the apps. The contributions of this work are twofold, first, it provides the fundamental knowledge on the building blocks of camera barcode reading apps, and second, it explores the issues in the current camera barcode reading apps that could encourage exploration towards addressing the issues. Practically, the findings could spark new research ideas to address the current issues related to the use of smartphone camera for barcode reading in the near future

Vision artificielle pour les non-voyants : une approche bio-inspirée pour la reconnaissance de formes

More than 315 million people worldwide suffer from visual impairments, with several studies suggesting that this number will double by 2030 due to the ageing of the population. To compensate for the loss of sight the current approaches consist of either specific aids designed to answer particular needs or generic systems such as neuroprostheses and sensory substitution devices. These holistic approaches, which try to restore vision as a whole, have been shown to be very inefficient in real life situations given the low resolution of output interfaces. To overcome these obstacles we propose the use of artificial vision in order to pre-process visual scenes and provide the user with relevant information. We have validated this approach through the development of a novel assistive device for the blind called Navig. Through shape recognition and spatialized sounds synthesis, this system allows users to locate and grab objects of interest. It also features navigational aids based on a new positioning method combining GPS, inertial sensors and the visual detection of geolocalized landmarks. To enhance the performance of the visual module we further developed, as part of this thesis, a bio-inspired pattern recognition algorithm which uses latency-based coding of visual information, oriented edge representations and a cascaded architecture combining detection at different resolutions.La déficience visuelle touche aujourd’hui plus de 315 millions de personnes à travers le monde, un chiffre qui pourrait doubler d’ici à 2030 du fait du vieillissement de la population. Les deux grandes approches existantes pour compenser la perte de vision sont les aides spécifiques, répondant à un besoin identifié, et les systèmes génériques tels que les neuroprothèses ou les systèmes de substitution sensorielle. Ces approches holistiques, tentant de restituer l’ensemble de l’information visuelle, s’avèrent inadaptées de par la trop faible résolution des interfaces de sortie, rendant ces systèmes inutilisables dans la vie quotidienne. Face à ce constat, nous proposons dans cette thèse une démarche alternative, consistant à intégrer des méthodes de vision artificielle, afin de prétraiter la scène visuelle, et de ne restituer au non-voyant que les informations extraites pertinentes. Pour valider cette approche, nous présenterons le développement d’un système de suppléance baptisé Navig. Grâce à la reconnaissance de formes et à la synthèse de sons spatialisés, il permet à l’utilisateur de localiser des objets d’intérêt. Il offre également des fonctions de navigation, basées sur une nouvelle méthode de positionnement combinant GPS, données inertielles, et détections de cibles visuelles géolocalisées. Afin d’améliorer les performances du module de vision artificielle, nous proposerons également dans cette thèse un nouvel algorithme de reconnaissance de formes bio-inspiré, reposant sur un codage de l’information visuelle par latence, sur des représentations sous forme d’arêtes orientées, et sur une architecture en cascade combinant des détections à différentes résolutions