Indoor navigation for the visually impaired : enhancements through utilisation of the Internet of Things and deep learning

Wayfinding and navigation are essential aspects of independent living that heavily rely on the sense of vision. Walking in a complex building requires knowing exact location to find a suitable path to the desired destination, avoiding obstacles and monitoring orientation and movement along the route. People who do not have access to sight-dependent information, such as that provided by signage, maps and environmental cues, can encounter challenges in achieving these tasks independently. They can rely on assistance from others or maintain their independence by using assistive technologies and the resources provided by smart environments. Several solutions have adapted technological innovations to combat navigation in an indoor environment over the last few years. However, there remains a significant lack of a complete solution to aid the navigation requirements of visually impaired (VI) people. The use of a single technology cannot provide a solution to fulfil all the navigation difficulties faced. A hybrid solution using Internet of Things (IoT) devices and deep learning techniques to discern the patterns of an indoor environment may help VI people gain confidence to travel independently. This thesis aims to improve the independence and enhance the journey of VI people in an indoor setting with the proposed framework, using a smartphone. The thesis proposes a novel framework, Indoor-Nav, to provide a VI-friendly path to avoid obstacles and predict the user s position. The components include Ortho-PATH, Blue Dot for VI People (BVIP), and a deep learning-based indoor positioning model. The work establishes a novel collision-free pathfinding algorithm, Orth-PATH, to generate a VI-friendly path via sensing a grid-based indoor space. Further, to ensure correct movement, with the use of beacons and a smartphone, BVIP monitors the movements and relative position of the moving user. In dark areas without external devices, the research tests the feasibility of using sensory information from a smartphone with a pre-trained regression-based deep learning model to predict the user s absolute position. The work accomplishes a diverse range of simulations and experiments to confirm the performance and effectiveness of the proposed framework and its components. The results show that Indoor-Nav is the first type of pathfinding algorithm to provide a novel path to reflect the needs of VI people. The approach designs a path alongside walls, avoiding obstacles, and this research benchmarks the approach with other popular pathfinding algorithms. Further, this research develops a smartphone-based application to test the trajectories of a moving user in an indoor environment

Factors affecting blind mobility

This thesis contains a survey of the mobility problems of blind people, experimental analysis and evaluation of these problems and suggestions for ways in which the evaluation of mobility performance and the design of mobility aids may be improved. The survey revealed a low level of mobility among blind people, with no significant improvement since a comparable survey in 1967. A group of self taught cane users were identified and their mobility was shown to be poor or potentially dangerous. Existing measures of mobility were unable to detect improvements in performance above that achieved by competent long cane users. By using newly devised measures of environmental awareness and of gait, the advantages of the Sonic Pathfinder were demonstrated. Existing measures of psychological stress were unsatisfactory. Heart rate is affected by physical effort and has been shown to be a poor indicator of moment-to-moment stress in blind mobility. Analysis of secondary task errors showed that they occurred while obstacles were being negotiated. They did not measure stress due to anticipation of obstacles or of danger. In contrast, step length, stride time and particularly speed all show significant anticipatory effects. The energy expended in walking a given distance is least at the walker's preferred speed. When guided, blind people walk at this most efficient pace. It is therefore suggested that the ratio of actual to preferred speed is the best measure of efficiency in mobility. Both guide dogs and aids which enhance preview allow pedestrians to walk at, or close to, their preferred speed. Further experiments are needed to establish the extent to which psychological stress is present during blind mobility, since none of the conventional measures, such as heart rate and mood checklists show consistent effects. Walking speed may well prove to be the most useful measure of such stress

Mind the Gap: Developments in Autonomous Driving Research and the Sustainability Challenge

Scientific knowledge on autonomous-driving technology is expanding at a faster-than-ever pace. As a result, the likelihood of incurring information overload is particularly notable for researchers, who can struggle to overcome the gap between information processing requirements and information processing capacity. We address this issue by adopting a multi-granulation approach to latent knowledge discovery and synthesis in large-scale research domains. The proposed methodology combines citation-based community detection methods and topic modeling techniques to give a concise but comprehensive overview of how the autonomous vehicle (AV) research field is conceptually structured. Thirteen core thematic areas are extracted and presented by mining the large data-rich environments resulting from 50 years of AV research. The analysis demonstrates that this research field is strongly oriented towards examining the technological developments needed to enable the widespread rollout of AVs, whereas it largely overlooks the wide-ranging sustainability implications of this sociotechnical transition. On account of these findings, we call for a broader engagement of AV researchers with the sustainability concept and we invite them to increase their commitment to conducting systematic investigations into the sustainability of AV deployment. Sustainability research is urgently required to produce an evidence-based understanding of what new sociotechnical arrangements are needed to ensure that the systemic technological change introduced by AV-based transport systems can fulfill societal functions while meeting the urgent need for more sustainable transport solutions

Advisory Safety System for Autonomous Vehicles under Sun-glare

Autonomous Vehicles (AVs) are expected to provide a large number of benefits such as improving comfort, vehicle safety and traffic flow. AVs use various sensors and control systems to empower driver’s decision-making under uncertainties as well as, assist the driving task under adverse conditions such as vision impairment. Excessive sunlight has been recognized as the primary source of the reduction in vision performance during daytime. Sun glare oftentimes leads to an impaired visibility for drivers and has been studied from different aspects on roadways. However, there is a lack of knowledge regarding the potential detrimental effects of natural light brightness differential, particularly sun glare on driving behavior and its possible risks. This dissertation addresses this issue by developing an integrated vehicle safety methodology as an advisory system for safe driving under sun glare. The main contribution of this research is to establish a real-time detection of the vision impairment area on roadways. This study also proposes a Collision Avoidance System Under Sun-glare (CASUS) in which upcoming possible vision impairment is detected, a warning message is sent, and the speed of vehicle is adjusted accordingly. In this context, real-world data is used to calibrate a psychophysical car-following model within VISSIM, a traffic microscopic simulation tool. Traffic safety impacts are explored through the number of conflicts extracted from the microsimulation tool and assessed by the time-to-collision indicator. Conventional/human-driven vehicles and different type of AVs are modeled for a straight segment of the TransCanada highway under various AVs penetration rates. The findings revealed a significant reduction in potential collisions due to adjustment of travel speed of AVs under the sun glare. The results also indicated that applying CASUS to the AVs with a failing sensory system improves traffic safety by providing optimal-safe speeds. Furthermore, the CASUS algorithm has the potential to be integrated into driving simulators or real vehicles to further evaluate and examine its benefits under different vision impairment scenarios

Advances in Human-Robot Interaction

Rapid advances in the field of robotics have made it possible to use robots not just in industrial automation but also in entertainment, rehabilitation, and home service. Since robots will likely affect many aspects of human existence, fundamental questions of human-robot interaction must be formulated and, if at all possible, resolved. Some of these questions are addressed in this collection of papers by leading HRI researchers

Assessment of Audio Interfaces for use in Smartphone Based Spatial Learning Systems for the Blind

Recent advancements in the field of indoor positioning and mobile computing promise development of smart phone based indoor navigation systems. Currently, the preliminary implementations of such systems only use visual interfaces—meaning that they are inaccessible to blind and low vision users. According to the World Health Organization, about 39 million people in the world are blind. This necessitates the need for development and evaluation of non-visual interfaces for indoor navigation systems that support safe and efficient spatial learning and navigation behavior. This thesis research has empirically evaluated several different approaches through which spatial information about the environment can be conveyed through audio. In the first experiment, blindfolded participants standing at an origin in a lab learned the distance and azimuth of target objects that were specified by four audio modes. The first three modes were perceptual interfaces and did not require cognitive mediation on the part of the user. The fourth mode was a non-perceptual mode where object descriptions were given via spatial language using clockface angles. After learning the targets through the four modes, the participants spatially updated the position of the targets and localized them by walking to each of them from two indirect waypoints. The results also indicate hand motion triggered mode to be better than the head motion triggered mode and comparable to auditory snapshot. In the second experiment, blindfolded participants learned target object arrays with two spatial audio modes and a visual mode. In the first mode, head tracking was enabled, whereas in the second mode hand tracking was enabled. In the third mode, serving as a control, the participants were allowed to learn the targets visually. We again compared spatial updating performance with these modes and found no significant performance differences between modes. These results indicate that we can develop 3D audio interfaces on sensor rich off the shelf smartphone devices, without the need of expensive head tracking hardware. Finally, a third study, evaluated room layout learning performance by blindfolded participants with an android smartphone. Three perceptual and one non-perceptual mode were tested for cognitive map development. As expected the perceptual interfaces performed significantly better than the non-perceptual language based mode in an allocentric pointing judgment and in overall subjective rating. In sum, the perceptual interfaces led to better spatial learning performance and higher user ratings. Also there is no significant difference in a cognitive map developed through spatial audio based on tracking user’s head or hand. These results have important implications as they support development of accessible perceptually driven interfaces for smartphones

A Hybrid Visual Control Scheme to Assist the Visually Impaired with Guided Reaching Tasks

In recent years, numerous researchers have been working towards adapting technology developed for robotic control to use in the creation of high-technology assistive devices for the visually impaired. These types of devices have been proven to help visually impaired people live with a greater degree of confidence and independence. However, most prior work has focused primarily on a single problem from mobile robotics, namely navigation in an unknown environment. In this work we address the issue of the design and performance of an assistive device application to aid the visually-impaired with a guided reaching task. The device follows an eye-in-hand, IBLM visual servoing configuration with a single camera and vibrotactile feedback to the user to direct guided tracking during the reaching task. We present a model for the system that employs a hybrid control scheme based on a Discrete Event System (DES) approach. This approach avoids significant problems inherent in the competing classical control or conventional visual servoing models for upper limb movement found in the literature. The proposed hybrid model parameterizes the partitioning of the image state-space that produces a variable size targeting window for compensatory tracking in the reaching task. The partitioning is created through the positioning of hypersurface boundaries within the state space, which when crossed trigger events that cause DES-controller state transition that enable differing control laws. A set of metrics encompassing, accuracy ($D$), precision ($\theta_{e}$), and overall tracking performance ($\psi$) are also proposed to quantity system performance so that the effect of parameter variations and alternate controller configurations can be compared. To this end, a prototype called \texttt{aiReach} was constructed and experiments were conducted testing the functional use of the system and other supporting aspects of the system behaviour using participant volunteers. Results are presented validating the system design and demonstrating effective use of a two parameter partitioning scheme that utilizes a targeting window with additional hysteresis region to filtering perturbations due to natural proprioceptive limitations for precise control of upper limb movement. Results from the experiments show that accuracy performance increased with the use of the dual parameter hysteresis target window model ($0.91 \leq D \leq 1$, $\mu(D)=0.9644$, $\sigma(D)=0.0172$) over the single parameter fixed window model ($0.82 \leq D \leq 0.98$, $\mu(D)=0.9205$, $\sigma(D)=0.0297$) while the precision metric, $\theta_{e}$, remained relatively unchanged. In addition, the overall tracking performance metric produces scores which correctly rank the performance of the guided reaching tasks form most difficult to easiest

Autonomous Robot Navigation through a Crowded and Dynamic Environment: Using A Novel form of Path Planning to Demonstrate Consideration towards Pedestrians and other Robots

This thesis presents a novel path planning algorithm for robotic crowd navigation through a pedestrian environment. The robot is designed to negotiate its way through the crowd using considerate movements. Unlike many other path planning algorithms, which assume cooperation with other pedestrians, this algorithm is completely independent and requires only observation. A considerate navigation strategy has been developed in this thesis, which utilises consideration as an directs an autonomous mobile robot. Using simple methods of predicting pedestrian movements, as well as simple relative distance and trajectory measurements between the robot and pedestrians, the robot can navigate through a crowd without causing disruption to pedestrian trajectories. Dynamic pedestrian positions are predicted using uncertainty ellipses. A novel Voronoi diagram-visibility graph hybrid roadmap is implemented so that the path planner can exploit any available gaps in between pedestrians, and plan considerate paths. The aim of the considerate path planner is to have the robot behave in specific ways when moving through the crowd. By predicting pedestrian trajectories, the robot can avoid interfering with them. Following preferences to move behind pedestrians, when cutting across their trajectories; to move in the same direction of the crowd when possible; and to slow down in crowded areas, will prevent any interference to individual pedestrians, as well as preventing an increase in congestion to the crowd as a whole. The effectiveness of the considerate navigation strategy is evaluated using simulated pedestrians, multiple mobile robots loaded with the path planning algorithm, as well as a real-life pedestrian dataset. The algorithm will highlight its ability to move with the aforementioned consideration towards each individual dynamic agent