SACSoN: Scalable Autonomous Data Collection for Social Navigation

Machine learning provides a powerful tool for building socially compliant robotic systems that go beyond simple predictive models of human behavior. By observing and understanding human interactions from past experiences, learning can enable effective social navigation behaviors directly from data. However, collecting navigation data in human-occupied environments may require teleoperation or continuous monitoring, making the process prohibitively expensive to scale. In this paper, we present a scalable data collection system for vision-based navigation, SACSoN, that can autonomously navigate around pedestrians in challenging real-world environments while encouraging rich interactions. SACSoN uses visual observations to observe and react to humans in its vicinity. It couples this visual understanding with continual learning and an autonomous collision recovery system that limits the involvement of a human operator, allowing for better dataset scaling. We use a this system to collect the SACSoN dataset, the largest-of-its-kind visual navigation dataset of autonomous robots operating in human-occupied spaces, spanning over 75 hours and 4000 rich interactions with humans. Our experiments show that collecting data with a novel objective that encourages interactions, leads to significant improvements in downstream tasks such as inferring pedestrian dynamics and learning socially compliant navigation behaviors. We make videos of our autonomous data collection system and the SACSoN dataset publicly available on our project page.Comment: 9 pages, 12 figures, 4 table

GNM: A General Navigation Model to Drive Any Robot

Learning provides a powerful tool for vision-based navigation, but the capabilities of learning-based policies are constrained by limited training data. If we could combine data from all available sources, including multiple kinds of robots, we could train more powerful navigation models. In this paper, we study how a general goal-conditioned model for vision-based navigation can be trained on data obtained from many distinct but structurally similar robots, and enable broad generalization across environments and embodiments. We analyze the necessary design decisions for effective data sharing across robots, including the use of temporal context and standardized action spaces, and demonstrate that an omnipolicy trained from heterogeneous datasets outperforms policies trained on any single dataset. We curate 60 hours of navigation trajectories from 6 distinct robots, and deploy the trained GNM on a range of new robots, including an underactuated quadrotor. We find that training on diverse data leads to robustness against degradation in sensing and actuation. Using a pre-trained navigation model with broad generalization capabilities can bootstrap applications on novel robots going forward, and we hope that the GNM represents a step in that direction. For more information on the datasets, code, and videos, please check out http://sites.google.com/view/drive-any-robot

ViNT: A Foundation Model for Visual Navigation

General-purpose pre-trained models ("foundation models") have enabled practitioners to produce generalizable solutions for individual machine learning problems with datasets that are significantly smaller than those required for learning from scratch. Such models are typically trained on large and diverse datasets with weak supervision, consuming much more training data than is available for any individual downstream application. In this paper, we describe the Visual Navigation Transformer (ViNT), a foundation model that aims to bring the success of general-purpose pre-trained models to vision-based robotic navigation. ViNT is trained with a general goal-reaching objective that can be used with any navigation dataset, and employs a flexible Transformer-based architecture to learn navigational affordances and enable efficient adaptation to a variety of downstream navigational tasks. ViNT is trained on a number of existing navigation datasets, comprising hundreds of hours of robotic navigation from a variety of different robotic platforms, and exhibits positive transfer, outperforming specialist models trained on singular datasets. ViNT can be augmented with diffusion-based subgoal proposals to explore novel environments, and can solve kilometer-scale navigation problems when equipped with long-range heuristics. ViNT can also be adapted to novel task specifications with a technique inspired by prompt-tuning, where the goal encoder is replaced by an encoding of another task modality (e.g., GPS waypoints or routing commands) embedded into the same space of goal tokens. This flexibility and ability to accommodate a variety of downstream problem domains establishes ViNT as an effective foundation model for mobile robotics. For videos, code, and model checkpoints, see our project page at https://visualnav-transformer.github.io.Comment: Accepted for oral presentation at CoRL 202

In-situ mechanical weakness of subducting sediments beneath a plate boundary décollement in the Nankai Trough

© 2018, The Author(s). The study investigates the in-situ strength of sediments across a plate boundary décollement using drilling parameters recorded when a 1180-m-deep borehole was established during International Ocean Discovery Program (IODP) Expedition 370, Temperature-Limit of the Deep Biosphere off Muroto (T-Limit). Information of the in-situ strength of the shallow portion in/around a plate boundary fault zone is critical for understanding the development of accretionary prisms and of the décollement itself. Studies using seismic reflection surveys and scientific ocean drillings have recently revealed the existence of high pore pressure zones around frontal accretionary prisms, which may reduce the effective strength of the sediments. A direct measurement of in-situ strength by experiments, however, has not been executed due to the difficulty in estimating in-situ stress conditions. In this study, we derived a depth profile for the in-situ strength of a frontal accretionary prism across a décollement from drilling parameters using the recently established equivalent strength (EST) method. At site C0023, the toe of the accretionary prism area off Cape Muroto, Japan, the EST gradually increases with depth but undergoes a sudden change at ~ 800 mbsf, corresponding to the top of the subducting sediment. At this depth, directly below the décollement zone, the EST decreases from ~ 10 to 2 MPa, with a change in the baseline. This mechanically weak zone in the subducting sediments extends over 250 m (~ 800–1050 mbsf), corresponding to the zone where the fluid influx was discovered, and high-fluid pressure was suggested by previous seismic imaging observations. Although the origin of the fluids or absolute values of the strength remain unclear, our investigations support previous studies suggesting that elevated pore pressure beneath the décollement weakens the subducting sediments. [Figure not available: see fulltext.]