Navigation performance effects of render method and latency in mobile audio augmented reality

Presented at the 15th International Conference on Auditory Display (ICAD2009), Copenhagen, Denmark, May 18-22, 2009This paper describes a pilot study and main experiment that as- sess user performance at navigating to spatialised sound sources using a mobile audio augmented reality system. Experiments use a novel outdoor paradigm with an application-relevant navigation task to compare perception of two binaural rendering methods un- der several head-turn latencies. Binaural rendering methods ex- amined were virtual, 6-speaker, first-order Ambisonic and virtual 12-speaker VBAP techniques. This study extends existing indoors research on the effects of head-turn latency for seated listeners. The pilot study examined the effect of capture radius (of 1, 2, 3, 4 and 5 metres) on mean distance efficiency for a single user’s navigation path to sound sources. A significant performance degradation was found to occur for radii of 2 m. The main exper- iment examined the effect of render method and total system la- tency to head-turns (176 ms minimum plus 0, 100, 200, 400 and 800 ms) on mean distance efficiency and subjective stability rating (on a scale of 1-5), for 8 participants. Render method significantly affected distance efficiency and 800 ms of added head-turn latency significantly affected subjective stability. Also, the study revealed a significant interaction effect of render method and head-turn la- tency: Ambisonic rendering didn’t significantly affect subjective stability due to added head-turn latency, while VBAP rendering did. Thus, it appears rendering method can mitigate or potentiate stability effects of head-turn latency. The study also exemplifies that the novel experimental paradigm is capable of revealing statis- tically significant performance differences between mobile audio AR implementations

Navigation performance in a virtual environment with bonephones

Presented at the 11th International Conference on Auditory Display (ICAD2005)Audio navigation interfaces have traditionally been studied (and implemented) using headphones. However, many potential users (especially those with visual impairments) are hesitant to adopt these emerging wayfinding technologies if doing so requires them to reduce their ability to hear environmental sounds by wearing headphones. In this study we examined the performance of the SWAN audio navigation interface using bone-conduction headphones (``bonephones''), which do not cover the ear. Bonephones enabled all participants to complete the navigation tasks with good efficiencies, though not immediately as effective as regular headphones. Given the functional success here, and considering that the spatialization routines were not optimized for bonephones (this essentially represents a worst-case scenario), the prospects are excellent for more widespread usage of bone conduction for auditory navigation, and likely for many other auditory displays

Taux : a system for evaluating sound feedback in navigational tasks

This thesis presents the design and development of an evaluation system for generating audio displays that provide feedback to persons performing navigation tasks. It first develops the need for such a system by describing existing wayfinding solutions, investigating new electronic location-based methods that have the potential of changing these solutions and examining research conducted on relevant audio information representation techniques. An evaluation system that supports the manipulation of two basic classes of audio display is then described. Based on prior work on wayfinding with audio display, research questions are developed that investigate the viability of different audio displays. These are used to generate hypotheses and develop an experiment which evaluates four variations of audio display for wayfinding. Questions are also formulated that evaluate a baseline condition that utilizes visual feedback. An experiment which tests these hypotheses on sighted users is then described. Results from the experiment suggest that spatial audio combined with spoken hints is the best approach of the approaches comparing spatial audio. The test experiment results also suggest that muting a varying audio signal when a subject is on course did not improve performance. The system and method are then refined. A second experiment is conducted with improved displays and an improved experiment methodology. After adding blindfolds for sighted subjects and increasing the difficulty of navigation tasks by reducing the arrival radius, similar comparisons were observed. Overall, the two experiments demonstrate the viability of the prototyping tool for testing and refining multiple different audio display combinations for navigational tasks. The detailed contributions of this work and future research opportunities conclude this thesis

Multimodal Content Delivery for Geo-services

This thesis describes a body of work carried out over several research projects in the area of multimodal interaction for location-based services. Research in this area has progressed from using simulated mobile environments to demonstrate the visual modality, to the ubiquitous delivery of rich media using multimodal interfaces (geo- services). To effectively deliver these services, research focused on innovative solutions to real-world problems in a number of disciplines including geo-location, mobile spatial interaction, location-based services, rich media interfaces and auditory user interfaces. My original contributions to knowledge are made in the areas of multimodal interaction underpinned by advances in geo-location technology and supported by the proliferation of mobile device technology into modern life. Accurate positioning is a known problem for location-based services, contributions in the area of mobile positioning demonstrate a hybrid positioning technology for mobile devices that uses terrestrial beacons to trilaterate position. Information overload is an active concern for location-based applications that struggle to manage large amounts of data, contributions in the area of egocentric visibility that filter data based on field-of-view demonstrate novel forms of multimodal input. One of the more pertinent characteristics of these applications is the delivery or output modality employed (auditory, visual or tactile). Further contributions in the area of multimodal content delivery are made, where multiple modalities are used to deliver information using graphical user interfaces, tactile interfaces and more notably auditory user interfaces. It is demonstrated how a combination of these interfaces can be used to synergistically deliver context sensitive rich media to users - in a responsive way - based on usage scenarios that consider the affordance of the device, the geographical position and bearing of the device and also the location of the device

NAVIGATION PERFORMANCE EFFECTS OF RENDER METHOD AND LATENCY IN MOBILE AUDIO AUGMENTED REALITY

This paper describes a pilot study and main experiment that assess user performance at navigating to spatialised sound sources using a mobile audio augmented reality system. Experiments use a novel outdoor paradigm with an application-relevant navigation task to compare perception of two binaural rendering methods under several head-turn latencies. Binaural rendering methods examined were virtual, 6-speaker, first-order Ambisonic and virtual 12-speaker VBAP techniques. This study extends existing indoors research on the effects of head-turn latency for seated listeners. The pilot study examined the effect of capture radius (of 1, 2, 3, 4 and 5 metres) on mean distance efficiency for a single user’s navigation path to sound sources. A significant performance degradation was found to occur for radii of 2 m. The main experiment examined the effect of render method and total system latency to head-turns (176 ms minimum plus 0, 100, 200, 400 and 800 ms) on mean distance efficiency and subjective stability rating (on a scale of 1-5), for 8 participants. Render method significantly affected distance efficiency and 800 ms of added head-turn latency significantly affected subjective stability. Also, the study revealed a significant interaction effect of render method and head-turn latency: Ambisonic rendering didn’t significantly affect subjective stability due to added head-turn latency, while VBAP rendering did. Thus, it appears rendering method can mitigate or potentiate stability effects of head-turn latency. The study also exemplifies that the novel experimental paradigm is capable of revealing statistically significant performance differences between mobile audio AR implementations

Diegetic Sonification for Low Vision Gamers

There are not many games designed for all players that provide accommodations for low vision users. This means that low vision users may not get to engage with the gaming community in the same way as their sighted peers. In this thesis, I explore how diegetic sonification can be used as a tool to support these low vision gamers in the typical gaming environment. I asked low vision players to engage with a prototype game level with two diegetic sonification techniques applied, without the use of their corrective lenses. I found that participants had more enjoyment and experienced less difficulty and confusion when the sonification techniques were applied. This shows that diegetic sonification can be used to support low vision players in playing games without their corrective lenses and improved their overall enjoyment.

The Influence of Collective Working Memory Strategies on Agent Teams

Past self-organizing models of collectively moving "particles" (simulated bird flocks, fish schools, etc.) typically have been based on purely reflexive agents that have no significant memory of past movements or environmental obstacles. These agent collectives usually operate in abstract environments, but as these domains take on a greater realism, the collective requires behaviors use not only presently observed stimuli but also remembered information. It is hypothesized that the addition of a limited working memory of the environment, distributed among the collective's individuals can improve efficiency in performing tasks. This is first approached in a more traditional particle system in an abstract environment. Then it is explored for a single agent, and finally a team of agents, operating in a simulated 3-dimensional environment of greater realism. In the abstract environment, a limited distributed working memory produced a significant improvement in travel between locations, in some cases improving performance over time, while in others surprisingly achieving an immediate benefit from the influence of memory. When strategies for accumulating and manipulating memory were subsequently explored for a more realistic single agent in the 3-dimensional environment, if the agent kept a local or a cumulative working memory, its performance improved on different tasks, both when navigating nearby obstacles and, in the case of cumulative memory, when covering previously traversed terrain. When investigating a team of these agents engaged in a pursuit scenario, it was determined that a communicating and coordinating team still benefited from a working memory of the environment distributed among the agents, even with limited memory capacity. This demonstrates that a limited distributed working memory in a multi-agent system improves performance on tasks in domains of increasing complexity. This is true even though individual agents know only a fraction of the collective's entire memory, using this partial memory and interactions with others in the team to perform tasks. These results may prove useful in improving existing methodologies for control of collective movements for robotic teams, computer graphics, particle swarm optimization, and computer games, and in interpreting future experimental research on group movements in biological populations

Autonomous Collision avoidance for Unmanned aerial systems

Unmanned Aerial System (UAS) applications are growing day by day and this will lead Unmanned Aerial Vehicle (UAV) in the close future to share the same airspace of manned aircraft.This implies the need for UAS to define precise safety standards compatible with operations standards for manned aviation. Among these standards the need for a Sense And Avoid (S&A) system to support and, when necessary, sub¬stitute the pilot in the detection and avoidance of hazardous situations (e.g. midair collision, controlled flight into terrain, flight path obstacles, and clouds). This thesis presents the work come out in the development of a S&A system taking into account collision risks scenarios with multiple moving and fixed threats. The conflict prediction is based on a straight projection of the threats state in the future. The approximations introduced by this approach have the advantage of high update frequency (1 Hz) of the estimated conflict geometry. This solution allows the algorithm to capture the trajectory changes of the threat or ownship. The resolution manoeuvre evaluation is based on a optimisation approach considering step command applied to the heading and altitude autopilots. The optimisation problem takes into account the UAV performances and aims to keep a predefined minimum separation distance between UAV and threats during the resolution manouvre. The Human-Machine Interface (HMI) of this algorithm is then embedded in a partial Ground Control Station (GCS) mock-up with some original concepts for the indication of the flight condition parameters and the indication of the resolution manoeuvre constraints. Simulations of the S&A algorithm in different critical scenarios are moreover in-cluded to show the algorithm capabilities. Finally, methodology and results of the tests and interviews with pilots regarding the proposed GCS partial layout are covered

A Novel Path Planning Optimization Algorithm for Semi-Autonomous UAV in Bird Repellent Systems Based in Particle Swarm Optimization

Bird damage to fruit crops causes significant monetary losses to farmers annually. The application of traditional bird repelling methods such as bird cannons and tree netting became inefficient in the long run, keeping high maintenance and reduced mobility. Due to their versatility, Unmanned Aerial Vehicles (UAVs) can be beneficial to solve this problem. However, due to their low battery capacity that equals low flight duration, it is necessary to evolve path planning optimization. A path planning optimization algorithm of UAVs based on Particle Swarm Optimization (PSO) is presented in this dissertation. This technique was used due to the need for an easy implementation optimization algorithm to start the initial tests. The PSO algorithm is simple and has few control parameters while maintaining a good performance. This path planning optimization algorithm aims to manage the drone's distance and flight time, applying optimization and randomness techniques to overcome the disadvantages of the traditional systems. The proposed algorithm's performance was tested in three study cases: two of them in simulation to test the variation of each parameter and one in the field to test the influence on battery management and height influence. All cases were tested in the three possible situations: same incidence rate, different rates, and different rates with no bird damage to fruit crops. The proposed algorithm presents promising results with an outstanding reduced average error in the total distance for the path planning obtained and low execution time. However, it is necessary to point out that the path planning optimization algorithm may have difficulty finding a suitable solution if there is a bad ratio between the total distance for path planning and points of interest. The field tests were also essential to understand the algorithm's behavior of the path planning algorithm in the UAV, showing that there is less energy discharged with fewer points of interest, but that do not correlates with the flight time. Also, there is no association between the maximum horizontal speed and the flight time, which means that the function to calculate the total distance for path planning needs to be adjusted.Anualmente, os danos causados pelas aves em pomares criam perdas monetárias significativas aos agricultores. A aplicação de métodos tradicionais de dispersão de aves, como canhões repelentes de aves e redes nas árvores, torna-se ineficiente a longo prazo, sendo ainda de alta manutenção e de mobilidade reduzida. Devido à sua versatilidade, os Veículos Aéreos Não Tripulados (VANT) podem ser benéficos para resolver este problema. No entanto, devido à baixa capacidade das suas baterias, que se traduz num baixo tempo de voo, é necessário otimizar o planeamento dos caminhos. Nesta dissertação, é apresentado um algoritmo de otimização para planeamento de caminhos para VANT baseado no Particle Swarm Optimization (PSO). Para se iniciarem os primeiros testes do algoritmo proposto, a técnica utilizada foi a supracitada devido à necessidade de um algoritmo de otimização fácil de implementar. O algoritmo PSO é simples e possuí poucos parâmetros de controlo, mantendo um bom desempenho. Este algoritmo de otimização de planeamento de caminhos propõe-se a gerir a distância e o tempo de voo do drone, aplicando técnicas de otimização e de aleatoriedade para superar a sua desvantagem relativamente aos sistemas tradicionais. O desempenho do algoritmo de planeamento de caminhos foi testado em três casos de estudo: dois deles em simulação para testar a variação de cada parâmetro e outro em campo para testar a capacidade da bateria. Todos os casos foram testados nas três situações possíveis: mesma taxa de incidência, taxas diferentes e taxas diferentes sem danos de aves. Os resultados apresentados pelo algoritmo proposto demonstram um erro médio muto reduzido na distância total para o planeamento de caminhos obtido e baixo tempo de execução. Porém, é necessário destacar que o algoritmo pode ter dificuldade em encontrar uma solução adequada se houver uma má relação entre a distância total para o planeamento de caminhos e os pontos de interesse. Os testes de campo também foram essenciais para entender o comportamento do algoritmo na prática, mostrando que há menos energia consumida com menos pontos de interesse, sendo que este parâmetro não se correlaciona com o tempo de voo. Além disso, não há associação entre a velocidade horizontal máxima e o tempo da missão, o que significa que a função de cálculo da distância total para o planeamento de caminhos requer ser ajustada

Assessment of Audio Interfaces for use in Smartphone Based Spatial Learning Systems for the Blind

Recent advancements in the field of indoor positioning and mobile computing promise development of smart phone based indoor navigation systems. Currently, the preliminary implementations of such systems only use visual interfaces—meaning that they are inaccessible to blind and low vision users. According to the World Health Organization, about 39 million people in the world are blind. This necessitates the need for development and evaluation of non-visual interfaces for indoor navigation systems that support safe and efficient spatial learning and navigation behavior. This thesis research has empirically evaluated several different approaches through which spatial information about the environment can be conveyed through audio. In the first experiment, blindfolded participants standing at an origin in a lab learned the distance and azimuth of target objects that were specified by four audio modes. The first three modes were perceptual interfaces and did not require cognitive mediation on the part of the user. The fourth mode was a non-perceptual mode where object descriptions were given via spatial language using clockface angles. After learning the targets through the four modes, the participants spatially updated the position of the targets and localized them by walking to each of them from two indirect waypoints. The results also indicate hand motion triggered mode to be better than the head motion triggered mode and comparable to auditory snapshot. In the second experiment, blindfolded participants learned target object arrays with two spatial audio modes and a visual mode. In the first mode, head tracking was enabled, whereas in the second mode hand tracking was enabled. In the third mode, serving as a control, the participants were allowed to learn the targets visually. We again compared spatial updating performance with these modes and found no significant performance differences between modes. These results indicate that we can develop 3D audio interfaces on sensor rich off the shelf smartphone devices, without the need of expensive head tracking hardware. Finally, a third study, evaluated room layout learning performance by blindfolded participants with an android smartphone. Three perceptual and one non-perceptual mode were tested for cognitive map development. As expected the perceptual interfaces performed significantly better than the non-perceptual language based mode in an allocentric pointing judgment and in overall subjective rating. In sum, the perceptual interfaces led to better spatial learning performance and higher user ratings. Also there is no significant difference in a cognitive map developed through spatial audio based on tracking user’s head or hand. These results have important implications as they support development of accessible perceptually driven interfaces for smartphones