Social Attention: Modeling Attention in Human Crowds

Robots that navigate through human crowds need to be able to plan safe, efficient, and human predictable trajectories. This is a particularly challenging problem as it requires the robot to predict future human trajectories within a crowd where everyone implicitly cooperates with each other to avoid collisions. Previous approaches to human trajectory prediction have modeled the interactions between humans as a function of proximity. However, that is not necessarily true as some people in our immediate vicinity moving in the same direction might not be as important as other people that are further away, but that might collide with us in the future. In this work, we propose Social Attention, a novel trajectory prediction model that captures the relative importance of each person when navigating in the crowd, irrespective of their proximity. We demonstrate the performance of our method against a state-of-the-art approach on two publicly available crowd datasets and analyze the trained attention model to gain a better understanding of which surrounding agents humans attend to, when navigating in a crowd

자율 주행을 위한 학습 기반의 다중 교통 참여자 경로예측 방법

학위논문 (석사)-- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2019. 2. 서승우.자율주행차량이 안전하면서도, 교통 흐름을 방해하지 않는 인간 수준의 interactive한 주행을 실현하기 위해서는 주변 운전자와 보행자를 포함한 교통 참여자들의 의도를 파악하는 것이 필수적이다. 동일한 도로 환경에서도 주변의 교통 참여자들이 어떤 의도를 가지고 이동하고 있는가에 따라서 적합한 주행 전략은 매번 달라진다. 특히 좁은 공간을 많은 에이전트들이 공유하고 있는 골목길 상황에서는 각 에이전트들의 선택은 제한되지만, 각 에이전트들의 의사결정과정은 그들간의 상호작용이 고려되어 매우 복잡하다.이런 상황에 맞는 경로예측은 과거의 궤적, 현재 인식하고 있는 도로 환경, 주변 교통 참여자의 상태 등을 고려하여 수행되어야 한다. 또한 일반화된 대부분의 환경에서 수행되려면, 빠른 러닝 타임과 동시에 고려하는 교통 참여자의 숫자에 대한 제약이 적어야 한다. 이 논문에서는 각 교통 참여자들의 과거 궤적과 현재 위치 상황, 도로 상황을 동시에 고려하여 모든 참여자들의 상호작용이 고려된 경로예측 방법을 제안한다. 각 물체들의 과거 궤적과 동시에 자율주행 차량이 인식하고 있는 도로와 주변 물체를 입력으로하고, 이 두개의 임베딩 결과를 혼합하여 모든 물체들에 대한 경로 예측을 동시에 수행한다. 이 과정에서 네트워크 구조 내부에서 각각 물체들의 임베딩 결과를 위치 정보에 매칭시킴으로써 효과적으로 주변 상황과 과거의 궤적을 동시에 고려하는 경로 예측을 학습할 수 있는 구조를 제안한다.몇가지 네트워크 구조에 따른 성능 비교와 함께, 다양한 주행 환경에서 정량적인 평가와 정성적인 평가로 유효성을 입증한다.또한 보행자 경로예측 데이터에 테스트를 진행함으로써 타 알고리즘과의 성능을 비교한다.Recent autonomous driving research has shown remarkable and promising results. However, safe, sociable driving in an urban environment still has many challenges ahead. For realizing safe, interactive driving in complex alley scenario which shares a narrow area among traffic participants, It is essential to grasp each other's intention. Even in the same road environment, safe, and sociable driving policy may differ depending on the intention of the traffic participant agents around the ego vehicle. But understanding others intention and predicting their trajectories are complicated because each one basically considers multiple factorsroad environment, state of their surrounding traffic participants at the same time which realized as interaction. In this thesis dissertation, we propose a new trajectory prediction algorithm that considers all the information that each of the traffic participants would consider when they make a decision. By combining both each of history trajectories and grid map of surroundings as a latent vector representation, it predicts all the future trajectories of traffic participant agents around ego vehicle at once. This dissertation suggests two main module that fuses spatial and temporal information effectively. We verify the effectiveness of network structure by testing on the various driving scenario comparing with some network variants through quantitative and qualitative evaluation. Also, the proposed network is verified by applying it to public pedestrian trajectory prediction dataset to verify usability as a generalized methodology and to compare it with other SOTA algorithms.Abstract Contents List of Tables List of Figures 1 Introduction 1.1 Background and Motivation 2 Related Work 2.1 Contributions of the Dissertation 3 Conditional Neural Process 3.1 Conditional Neural Process(CNP) Overview 3.2 Trajectory Prediction with Scene Information as CNP 3.2.1 Formulation 3.2.2 Loss and Training Algorithm 4 Efficient Network Architecture for Intention Prediction 4.1 Network Overview 4.2 Trajectory Encoder 4.2.1 Spatio-Temporal Representation 4.3 Scene Feature Extraction 4.3.1 Side Spatial Extraction 4.4 Trajectory Decoder 5 Experiment 5.1 Driving environment dataset 5.1.1 Data acquisition method 5.1.2 Overview 5.1.3 Alley scenario 5.1.4 Urban Scenario 5.2 Public Pedestrian Dataset 6 ConclusionMaste

Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page

Dynamic Occupancy Grid Prediction for Urban Autonomous Driving: A Deep Learning Approach with Fully Automatic Labeling

Long-term situation prediction plays a crucial role in the development of intelligent vehicles. A major challenge still to overcome is the prediction of complex downtown scenarios with multiple road users, e.g., pedestrians, bikes, and motor vehicles, interacting with each other. This contribution tackles this challenge by combining a Bayesian filtering technique for environment representation, and machine learning as long-term predictor. More specifically, a dynamic occupancy grid map is utilized as input to a deep convolutional neural network. This yields the advantage of using spatially distributed velocity estimates from a single time step for prediction, rather than a raw data sequence, alleviating common problems dealing with input time series of multiple sensors. Furthermore, convolutional neural networks have the inherent characteristic of using context information, enabling the implicit modeling of road user interaction. Pixel-wise balancing is applied in the loss function counteracting the extreme imbalance between static and dynamic cells. One of the major advantages is the unsupervised learning character due to fully automatic label generation. The presented algorithm is trained and evaluated on multiple hours of recorded sensor data and compared to Monte-Carlo simulation