5,609 research outputs found

    Enhancing environmental engagement with natural language interfaces for in-vehicle navigation systems

    Get PDF
    Four on-road studies were conducted in the Clifton area of Nottingham, UK, aiming to explore the relationships between driver workload and environmental engagement associated with ‘active’ and ‘passive’ navigation systems. In a between-subjects design, a total of 61 experienced drivers completed two experimental drives comprising the same three routes (with overlapping sections), staged one week apart. Drivers were provided with the navigational support of a commercially-available navigation device (‘satnav’), an informed passenger (a stranger with expert route knowledge), a collaborative passenger (an individual with whom they had a close, personal relationship) or a novel interface employing conversational natural language NAV-NLI). The NAV-NLI was created by curating linguistic intercourse extracted from the earlier conditions, and delivering this using a Wizard-of-Oz technique. The different navigational methods were notable for their varying interactivity and the preponderance of environmental landmark information within route directions. Participants experienced the same guidance on each of the two drives to explore changes in reported and observed behaviour. Results show that participants who were more active in the navigation task (collaborative passenger or NAV-NLI) demonstrated enhanced environmental engagement (landmark recognition, route-learning and survey knowledge) allowing them to reconstruct the route more accurately post-drive, compared to drivers using more passive forms of navigational support (SatNav or informed passenger). Workload measures (TDT, NASA-TLX) indicated no differences between conditions, although satnav users and collaborative passenger drivers reported lower workload during their second drive. The research demonstrates clear benefits and potential for a navigation system employing two-way conversational language to deliver instructions. This could help support a long-term perspective in the development of spatial knowledge, enabling drivers to become less reliant on the technology and begin to re-establish associations between viewing an environmental feature and the related navigational manoeuvre

    Sharing Human-Generated Observations by Integrating HMI and the Semantic Sensor Web

    Get PDF
    Current “Internet of Things” concepts point to a future where connected objects gather meaningful information about their environment and share it with other objects and people. In particular, objects embedding Human Machine Interaction (HMI), such as mobile devices and, increasingly, connected vehicles, home appliances, urban interactive infrastructures, etc., may not only be conceived as sources of sensor information, but, through interaction with their users, they can also produce highly valuable context-aware human-generated observations. We believe that the great promise offered by combining and sharing all of the different sources of information available can be realized through the integration of HMI and Semantic Sensor Web technologies. This paper presents a technological framework that harmonizes two of the most influential HMI and Sensor Web initiatives: the W3C’s Multimodal Architecture and Interfaces (MMI) and the Open Geospatial Consortium (OGC) Sensor Web Enablement (SWE) with its semantic extension, respectively. Although the proposed framework is general enough to be applied in a variety of connected objects integrating HMI, a particular development is presented for a connected car scenario where drivers’ observations about the traffic or their environment are shared across the Semantic Sensor Web. For implementation and evaluation purposes an on-board OSGi (Open Services Gateway Initiative) architecture was built, integrating several available HMI, Sensor Web and Semantic Web technologies. A technical performance test and a conceptual validation of the scenario with potential users are reported, with results suggesting the approach is soun

    Conceptual Model for an Intelligent Persuasive Driver Assistant

    Get PDF
    Traffic congestion is a serious issue for large cities.  This is especially critical for cities that has insufficient mass transit system like Bangkok.  Although transportation infrastructure projects and rail mass transit lines are being implemented, these efforts require major financial investment and take a long time to complete.  This work proposes to help reduce traffic problems through influencing a change in driver behavior.  In this initial stage, a model for an intelligent persuasive driver assistant is conceptualized as a voice-interactive smart assistant on a smartphone.  The system uses information about the driver, his physical state, vehicle performance information, and geolocation information to form persuasive strategies to influence driver behavior and to adapt user interfaces and interactions to reduce driver distraction.  Integrating these components together is expected to provide improved assistance in driving tasks and affect driving behavior changes. Keywords: intelligent driver assistant, navigation, smart assistant, persuasive technolog

    SGGNet2^2: Speech-Scene Graph Grounding Network for Speech-guided Navigation

    Full text link
    The spoken language serves as an accessible and efficient interface, enabling non-experts and disabled users to interact with complex assistant robots. However, accurately grounding language utterances gives a significant challenge due to the acoustic variability in speakers' voices and environmental noise. In this work, we propose a novel speech-scene graph grounding network (SGGNet2^2) that robustly grounds spoken utterances by leveraging the acoustic similarity between correctly recognized and misrecognized words obtained from automatic speech recognition (ASR) systems. To incorporate the acoustic similarity, we extend our previous grounding model, the scene-graph-based grounding network (SGGNet), with the ASR model from NVIDIA NeMo. We accomplish this by feeding the latent vector of speech pronunciations into the BERT-based grounding network within SGGNet. We evaluate the effectiveness of using latent vectors of speech commands in grounding through qualitative and quantitative studies. We also demonstrate the capability of SGGNet2^2 in a speech-based navigation task using a real quadruped robot, RBQ-3, from Rainbow Robotics.Comment: 7 pages, 6 figures, Paper accepted for the Special Session at the 2023 International Symposium on Robot and Human Interactive Communication (RO-MAN), [Dohyun Kim, Yeseung Kim, Jaehwi Jang, and Minjae Song] contributed equally to this wor

    Ambient awareness on a sidewalk for visually impaired

    Get PDF
    Safe navigation by avoiding obstacles is vital for visually impaired while walking on a sidewalk. There are both static and dynamic obstacles to avoid. Detection, monitoring, and estimating the threat posed by obstacles remain challenging. Also, it is imperative that the design of the system must be energy efficient and low cost. An additional challenge in designing an interactive system capable of providing useful feedback is to minimize users\u27 cognitive load. We started the development of the prototype system through classifying obstacles and providing feedback. To overcome the limitations of the classification-based system, we adopted the image annotation framework in describing the scene, which may or may not include the obstacles. Both solutions partially solved the safe navigation but were found to be ineffective in providing meaningful feedback and issues with the diurnal cycle. To address such limitations, we introduce the notion of free-path and threat level imposed by the static or dynamic obstacles. This solution reduced the overhead of obstacle detection and helped in designing meaningful feedback. Affording users a natural conversation through an interactive dialog enabled interface was found to promote safer navigation. In this dissertation, we modeled the free-path and threat level using a reinforcement learning (RL) framework.We built the RL model in the Gazebo robot simulation environment and implanted that in a handheld device. A natural conversation model was created using data collected through a Wizard of OZ approach. The RL model and conversational agent model together resulted in the handheld assistive device called Augmented Guiding Torch (AGT). The AGT provides improved mobility over white cane by providing ambient awareness through natural conversation. It can inform the visually impaired about the obstacles which are helpful to be warned about ahead of time, e.g., construction site, scooter, crowd, car, bike, or big hole. Using the RL framework, the robot avoided over 95% obstacles. The visually impaired avoided over 85% obstacles with the help of AGT on a 500 feet U-shape sidewalk. Findings of this dissertation support the effectiveness of augmented guiding through RL for navigation and obstacle avoidance of visually impaired users
    • 

    corecore