Explainable and Advisable Learning for Self-driving Vehicles

Deep neural perception and control networks are likely to be a key component of self-driving vehicles. These models need to be explainable - they should provide easy-to-interpret rationales for their behavior - so that passengers, insurance companies, law enforcement, developers, etc., can understand what triggered a particular behavior. Explanations may be triggered by the neural controller, namely introspective explanations, or informed by the neural controller's output, namely rationalizations. Our work has focused on the challenge of generating introspective explanations of deep models for self-driving vehicles. In Chapter 3, we begin by exploring the use of visual explanations. These explanations take the form of real-time highlighted regions of an image that causally influence the network's output (steering control). In the first stage, we use a visual attention model to train a convolution network end-to-end from images to steering angle. The attention model highlights image regions that potentially influence the network's output. Some of these are true influences, but some are spurious. We then apply a causal filtering step to determine which input regions actually influence the output. This produces more succinct visual explanations and more accurately exposes the network's behavior. In Chapter 4, we add an attention-based video-to-text model to produce textual explanations of model actions, e.g. "the car slows down because the road is wet". The attention maps of controller and explanation model are aligned so that explanations are grounded in the parts of the scene that mattered to the controller. We explore two approaches to attention alignment, strong- and weak-alignment. These explainable systems represent an externalization of tacit knowledge. The network's opaque reasoning is simplified to a situation-specific dependence on a visible object in the image. This makes them brittle and potentially unsafe in situations that do not match training data. In Chapter 5, we propose to address this issue by augmenting training data with natural language advice from a human. Advice includes guidance about what to do and where to attend. We present the first step toward advice-giving, where we train an end-to-end vehicle controller that accepts advice. The controller adapts the way it attends to the scene (visual attention) and the control (steering and speed). Further, in Chapter 6, we propose a new approach that learns vehicle control with the help of long-term (global) human advice. Specifically, our system learns to summarize its visual observations in natural language, predict an appropriate action response (e.g. "I see a pedestrian crossing, so I stop"), and predict the controls, accordingly

The design-by-adaptation approach to universal access: learning from videogame technology

This paper proposes an alternative approach to the design of universally accessible interfaces to that provided by formal design frameworks applied ab initio to the development of new software. This approach, design-byadaptation, involves the transfer of interface technology and/or design principles from one application domain to another, in situations where the recipient domain is similar to the host domain in terms of modelled systems, tasks and users. Using the example of interaction in 3D virtual environments, the paper explores how principles underlying the design of videogame interfaces may be applied to a broad family of visualization and analysis software which handles geographical data (virtual geographic environments, or VGEs). One of the motivations behind the current study is that VGE technology lags some way behind videogame technology in the modelling of 3D environments, and has a less-developed track record in providing the variety of interaction methods needed to undertake varied tasks in 3D virtual worlds by users with varied levels of experience. The current analysis extracted a set of interaction principles from videogames which were used to devise a set of 3D task interfaces that have been implemented in a prototype VGE for formal evaluation

Hybrid Satellite-Terrestrial Communication Networks for the Maritime Internet of Things: Key Technologies, Opportunities, and Challenges

With the rapid development of marine activities, there has been an increasing number of maritime mobile terminals, as well as a growing demand for high-speed and ultra-reliable maritime communications to keep them connected. Traditionally, the maritime Internet of Things (IoT) is enabled by maritime satellites. However, satellites are seriously restricted by their high latency and relatively low data rate. As an alternative, shore & island-based base stations (BSs) can be built to extend the coverage of terrestrial networks using fourth-generation (4G), fifth-generation (5G), and beyond 5G services. Unmanned aerial vehicles can also be exploited to serve as aerial maritime BSs. Despite of all these approaches, there are still open issues for an efficient maritime communication network (MCN). For example, due to the complicated electromagnetic propagation environment, the limited geometrically available BS sites, and rigorous service demands from mission-critical applications, conventional communication and networking theories and methods should be tailored for maritime scenarios. Towards this end, we provide a survey on the demand for maritime communications, the state-of-the-art MCNs, and key technologies for enhancing transmission efficiency, extending network coverage, and provisioning maritime-specific services. Future challenges in developing an environment-aware, service-driven, and integrated satellite-air-ground MCN to be smart enough to utilize external auxiliary information, e.g., sea state and atmosphere conditions, are also discussed

Survey of Inter-satellite Communication for Small Satellite Systems: Physical Layer to Network Layer View

Small satellite systems enable whole new class of missions for navigation, communications, remote sensing and scientific research for both civilian and military purposes. As individual spacecraft are limited by the size, mass and power constraints, mass-produced small satellites in large constellations or clusters could be useful in many science missions such as gravity mapping, tracking of forest fires, finding water resources, etc. Constellation of satellites provide improved spatial and temporal resolution of the target. Small satellite constellations contribute innovative applications by replacing a single asset with several very capable spacecraft which opens the door to new applications. With increasing levels of autonomy, there will be a need for remote communication networks to enable communication between spacecraft. These space based networks will need to configure and maintain dynamic routes, manage intermediate nodes, and reconfigure themselves to achieve mission objectives. Hence, inter-satellite communication is a key aspect when satellites fly in formation. In this paper, we present the various researches being conducted in the small satellite community for implementing inter-satellite communications based on the Open System Interconnection (OSI) model. This paper also reviews the various design parameters applicable to the first three layers of the OSI model, i.e., physical, data link and network layer. Based on the survey, we also present a comprehensive list of design parameters useful for achieving inter-satellite communications for multiple small satellite missions. Specific topics include proposed solutions for some of the challenges faced by small satellite systems, enabling operations using a network of small satellites, and some examples of small satellite missions involving formation flying aspects.Comment: 51 pages, 21 Figures, 11 Tables, accepted in IEEE Communications Surveys and Tutorial

Poznawcze przesłanki semiozy zorientowanej na mit

This article addresses the cognitive premises of designation units denoting mythic concepts in a variety of texts and discourses. The article focuses on myth-oriented semiosis as a cognitive and cultural phenomenon reflected in the semantic transformations of lingual signs, resulting in the development of noematic senses relevant to the states of affairs in diverse worldviews or modelled alternative realities. This article provides an analysis of the basic cognitive models and procedures responsible for irrational cognition. The reconstructed cognitive models are then discussed in terms of their correspondence with the universal patterns of open system interaction and information exchange.Ten artykuł dotyczy poznawczych przesłanek jednostek desygnacyjnych oznaczających mityczne pojęcia w różnych tekstach i dyskursach. Artykuł koncentruje się na semiozie zorientowanej na mit jako zjawisku poznawczym i kulturowym odzwierciedlonym w semantycznych przekształceniach znaków językowych, co skutkuje rozwojem noematycznych zmysłów związanych ze stanami rzeczy w różnych światopoglądach lub modelowanych alternatywnych rzeczywistościach. Ten artykuł zawiera analizę podstawowych modeli i procedur poznawczych odpowiedzialnych za irracjonalne poznanie. Zrekonstruowane modele poznawcze są następnie omawiane pod kątem ich zgodności z uniwersalnymi wzorcami interakcji otwartego systemu i wymiany informacji

Connected and Automated Vehicle Based Intersection Maneuver Assist Systems (CAVIMAS) and Their Impact on Driver Behavior, Acceptance, and Safety

Intersection crashes can be potentially mitigated by leveraging deployments of vehicle-to-infrastructure (V2I) and vehicle-to- vehicle (V2V) safety management solutions. However, it is equally critical that these deployments are undertaken in tandem with interventions based on human factors evidence relating to the content and presentation of such solutions. This driving simulator study designed and evaluated a conceptual system - Connected and Automated Vehicle based Intersection Maneuver Assist Systems (CAVIMAS) - aimed at assisting drivers with intersection maneuvers by leveraging connected infrastructure and providing real-time guidance and warnings and active vehicle controls. Results indicate that human factors considerations for the design and deployment of such systems remain paramount, given the findings related to drivers’ trust and acceptance of these systems as measured via surveys and by examining actual driving behaviors.Center for Connected and Automated Transportationhttps://deepblue.lib.umich.edu/bitstream/2027.42/156048/4/Connected_and_Automated_Vehicle_Based_Intersection_Maneuver_Assist_Systems_CAVIMAS.pd

Synchronous-clock range-angle relative acoustic navigation: a unified approach to multi-AUV localization, command, control, and coordination

© The Author(s), 2022. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Rypkema, N., Schmidt, H., & Fischell, E. Synchronous-clock range-angle relative acoustic navigation: a unified approach to multi-AUV localization, command, control, and coordination. Journal of Field Robotics, 2(1), (2022): 774–806, https://doi.org/10.55417/fr.2022026.This paper presents a scalable acoustic navigation approach for the unified command, control, and coordination of multiple autonomous underwater vehicles (AUVs). Existing multi-AUV operations typically achieve coordination manually by programming individual vehicles on the surface via radio communications, which becomes impractical with large vehicle numbers; or they require bi-directional intervehicle acoustic communications to achieve limited coordination when submerged, with limited scalability due to the physical properties of the acoustic channel. Our approach utilizes a single, periodically broadcasting beacon acting as a navigation reference for the group of AUVs, each of which carries a chip-scale atomic clock and fixed ultrashort baseline array of acoustic receivers. One-way travel-time from synchronized clocks and time-delays between signals received by each array element allow any number of vehicles within receive distance to determine range, angle, and thus determine their relative position to the beacon. The operator can command different vehicle behaviors by selecting between broadcast signals from a predetermined set, while coordination between AUVs is achieved without intervehicle communication by defining individual vehicle behaviors within the context of the group. Vehicle behaviors are designed within a beacon-centric moving frame of reference, allowing the operator to control the absolute position of the AUV group by repositioning the navigation beacon to survey the area of interest. Multiple deployments with a fleet of three miniature, low-cost SandShark AUVs performing closed-loop acoustic navigation in real-time provide experimental results validated against a secondary long-baseline positioning system, demonstrating the capabilities and robustness of our approach with real-world data.This work was partially supported by the Office of Naval Research, the Defense Advanced Research Projects Agency, Lincoln Laboratory, and the Reuben F. and Elizabeth B. Richards Endowed Funds at WHOI