Neural Sensor Fusion for Spatial Visualization on a Mobile Robot

An ARTMAP neural network is used to integrate visual information and ultrasonic sensory information on a B 14 mobile robot. Training samples for the neural network are acquired without human intervention. Sensory snapshots are retrospectively associated with the distance to the wall, provided by on~ board odomctry as the robot travels in a straight line. The goal is to produce a more accurate measure of distance than is provided by the raw sensors. The neural network effectively combines sensory sources both within and between modalities. The improved distance percept is used to produce occupancy grid visualizations of the robot's environment. The maps produced point to specific problems of raw sensory information processing and demonstrate the benefits of using a neural network system for sensor fusion.Office of Naval Research and Naval Research Laboratory (00014-96-1-0772, 00014-95-1-0409, 00014-95-0657

A principled approach to the measurement of situation awareness in commercial aviation

The issue of how to support situation awareness among crews of modern commercial aircraft is becoming especially important with the introduction of automation in the form of sophisticated flight management computers and expert systems designed to assist the crew. In this paper, cognitive theories are discussed that have relevance for the definition and measurement of situation awareness. These theories suggest that comprehension of the flow of events is an active process that is limited by the modularity of attention and memory constraints, but can be enhanced by expert knowledge and strategies. Three implications of this perspective for assessing and improving situation awareness are considered: (1) Scenario variations are proposed that tax awareness by placing demands on attention; (2) Experimental tasks and probes are described for assessing the cognitive processes that underlie situation awareness; and (3) The use of computer-based human performance models to augment the measures of situation awareness derived from performance data is explored. Finally, two potential example applications of the proposed assessment techniques are described, one concerning spatial awareness using wide field of view displays and the other emphasizing fault management in aircraft systems

Augmented Reality on Mobile Devices to Improve the Academic Achievement and Independence of Students with Disabilities

Augmented reality (AR) is a technology that overlays digital information on a live view of the physical world to create a blended experience. AR can provide unique experiences and opportunities to learn and interact with information in the physical world (Craig, 2013). The purpose of this dissertation was to investigate uses of AR on mobile devices to improve the academic and functional skills of students with disabilities. The first chapter is a literature review providing a clear understanding of AR and its connections with existing learning theories and evidence-based practices that are relevant for meeting the needs of individuals with disabilities. This chapter explores the available research on mobile devices, AR educational applications, and AR research involving students with disabilities. The purpose of Study 1 was to examine the effects of an augmented reality vocabulary instruction for science terms on college-aged students with ID. A multiple probe across skills design was used to determine if there was a functional relation between the AR vocabulary instruction and the acquisition of correctly defined and labeled science terms. The results indicated that all participants learned new science vocabulary terms using the augmented reality vocabulary instruction. Study 2 examined the effects of using an AR navigation, Google Maps, and a paper map as navigation aids for four college-aged students with ID enrolled in a PSE program. Using an adapted alternating treatments design, students used the three navigation aids to travel independently to unknown businesses in a large downtown city to seek employment opportunities. During the intervention phase, students used a mobile device with Google maps and the AR application to navigate to unfamiliar businesses. Results from Study 2 indicated all students improved navigation decision making when using AR. In the final chapter, both studies are discussed in relation to the AR research literature and as potential interventions. Findings from the studies include the capabilities of ARon mobile devices, academic and functional applications of this technology for students with disabilities, implications for mobile learning, and limitations of this technology. Recommendations for future research are presented to further examine using AR for students with disabilities

Conceptual design for Mobile Geological Laboratory position and heading fix system

Conceptual design of position fixing system for Mobile Geological Laboratory in Lunar Mobile Laboratory simulatio

From cognitive maps to spatial schemas

A schema refers to a structured body of prior knowledge that captures common patterns across related experiences. Schemas have been studied separately in the realms of episodic memory and spatial navigation across different species and have been grounded in theories of memory consolidation, but there has been little attempt to integrate our understanding across domains, particularly in humans. We propose that experiences during navigation with many similarly structured environments give rise to the formation of spatial schemas (for example, the expected layout of modern cities) that share properties with but are distinct from cognitive maps (for example, the memory of a modern city) and event schemas (such as expected events in a modern city) at both cognitive and neural levels. We describe earlier theoretical frameworks and empirical findings relevant to spatial schemas, along with more targeted investigations of spatial schemas in human and non-human animals. Consideration of architecture and urban analytics, including the influence of scale and regionalization, on different properties of spatial schemas may provide a powerful approach to advance our understanding of spatial schemas

Iterative Design and Prototyping of Computer Vision Mediated Remote Sighted Assistance

Remote sighted assistance (RSA) is an emerging navigational aid for people with visual impairments (PVI). Using scenario-based design to illustrate our ideas, we developed a prototype showcasing potential applications for computer vision to support RSA interactions. We reviewed the prototype demonstrating real-world navigation scenarios with an RSA expert, and then iteratively refined the prototype based on feedback. We reviewed the refined prototype with 12 RSA professionals to evaluate the desirability and feasibility of the prototyped computer vision concepts. The RSA expert and professionals were engaged by, and reacted insightfully and constructively to the proposed design ideas. We discuss what we learned about key resources, goals, and challenges of the RSA prosthetic practice through our iterative prototype review, as well as implications for the design of RSA systems and the integration of computer vision technologies into RSA

Development and usability analysis of a mixed reality GPS navigator application for the microsoft hololens

The present work aims to perform a comparative usability analysis between two Human- Computer Interaction systems (HCI) for global geolocation (GPS) navigators. The intent is to compare the conventional use of a navigation application on a mobile device, such as a smartphone attached to the dashboard of a vehicle, to an implementation in Mixed Reality (MR) powered by the Head Mounted Display (HMD) Microsoft HoloLens. By connecting the MR device to a local network routed by an ordinary cellular phone, which is connected to a mobile data network, it is possible to ubiquitously acquire the phone’s geolocation data, its magnetometer deviation and a route graph of a navigation Application Programming Interface (API) from its current location to a destination entered by the user. Thus, a series of three-dimensional holograms are created at runtime, geolocated and placed around the user, guiding him through a path indicated on the floor, pertinent to the streets around him that lead to the desired destination. Apart from that, arrows are projected on the way at each crucial point of the path, where some maneuver must be performed, e.g., turning right or taking an exit at a roundabout. In a user experiment, performance and usability were assessed. Results show that users deemed the MR solution to offer a higher visibility both to the oncoming traffic and the suggested route, when compared to the conventional interface, being less attention demanding. EEG readings for most participants also exposed a significantly more demanding focus level for the handheld device. Additionally, an easiness to learn and use was indicated for our system, being almost on par with the already known and highly used application tested.O presente trabalho visa realizar uma análise comparativa de usabilidade entre dois sistemas de interação humano-computador para navegadores de geolocalização global (GPS). Foi almejado comparar o uso convencional do sistema, através de um dispositivo móvel tal qual um smartphone afixado ao painel de um veículo, com uma nova implementação em Realidade Mista potencializada pelo HMD Microsoft HoloLens. Conectando o dispositivo de realidade mista (MR) a uma rede local roteada por um aparelho celular convencional, este conectado a uma rede de dados móvel, foi possível receber ubiquamente os dados de sua geolocalização, de seu magnetômetro e um grafo de rota de uma API de navegação de alta disponibilidade partindo do presente local até um destino inserido pelo usuário. Com isso, é criada em tempo de execução uma série de hologramas tridimensionais geolocalizados ao redor do usuário, guiando-o através de um caminho indicado em seu chão, pertinente às ruas a sua volta que o levarão ao destino desejado. Também são projetadas flechas em seu caminho em cada ponto crucial de seu trajeto, onde deve-se realizar alguma manobra, e.g., dobrar à direita ou tomar uma saída de uma rotatória. Em um experimento realizado com usuários reais, seu desempenho e usabilidade foram aferidos. Resultados mostram que os usuários estimaram que a solução em MR oferecia uma visibilidade maior tanto ao tráfego passante quanto à rota sugerida, em comparação à interface convencional, requerindo menos atenção. Leituras de eletroencefalografia (EEG) na maioria dos participantes indicaram uma demanda significativamente maior de atenção focada no uso do dispositivo móvel. Uma grande facilidade de aprendizado e de uso também foi apontada para nosso sistema, estando quase a par da aplicação móvel altamente conhecida e usada

Evaluating indoor positioning systems in a shopping mall : the lessons learned from the IPIN 2018 competition

The Indoor Positioning and Indoor Navigation (IPIN) conference holds an annual competition in which indoor localization systems from different research groups worldwide are evaluated empirically. The objective of this competition is to establish a systematic evaluation methodology with rigorous metrics both for real-time (on-site) and post-processing (off-site) situations, in a realistic environment unfamiliar to the prototype developers. For the IPIN 2018 conference, this competition was held on September 22nd, 2018, in Atlantis, a large shopping mall in Nantes (France). Four competition tracks (two on-site and two off-site) were designed. They consisted of several 1 km routes traversing several floors of the mall. Along these paths, 180 points were topographically surveyed with a 10 cm accuracy, to serve as ground truth landmarks, combining theodolite measurements, differential global navigation satellite system (GNSS) and 3D scanner systems. 34 teams effectively competed. The accuracy score corresponds to the third quartile (75th percentile) of an error metric that combines the horizontal positioning error and the floor detection. The best results for the on-site tracks showed an accuracy score of 11.70 m (Track 1) and 5.50 m (Track 2), while the best results for the off-site tracks showed an accuracy score of 0.90 m (Track 3) and 1.30 m (Track 4). These results showed that it is possible to obtain high accuracy indoor positioning solutions in large, realistic environments using wearable light-weight sensors without deploying any beacon. This paper describes the organization work of the tracks, analyzes the methodology used to quantify the results, reviews the lessons learned from the competition and discusses its future

A Neural Model of How the Brain Computes Heading from Optic Flow in Realistic Scenes

Animals avoid obstacles and approach goals in novel cluttered environments using visual information, notably optic flow, to compute heading, or direction of travel, with respect to objects in the environment. We present a neural model of how heading is computed that describes interactions among neurons in several visual areas of the primate magnocellular pathway, from retina through V1, MT+, and MSTd. The model produces outputs which are qualitatively and quantitatively similar to human heading estimation data in response to complex natural scenes. The model estimates heading to within 1.5° in random dot or photo-realistically rendered scenes and within 3° in video streams from driving in real-world environments. Simulated rotations of less than 1 degree per second do not affect model performance, but faster simulated rotation rates deteriorate performance, as in humans. The model is part of a larger navigational system that identifies and tracks objects while navigating in cluttered environments.National Science Foundation (SBE-0354378, BCS-0235398); Office of Naval Research (N00014-01-1-0624); National-Geospatial Intelligence Agency (NMA201-01-1-2016