Pedestrian Prediction by Planning using Deep Neural Networks

Accurate traffic participant prediction is the prerequisite for collision avoidance of autonomous vehicles. In this work, we predict pedestrians by emulating their own motion planning. From online observations, we infer a mixture density function for possible destinations. We use this result as the goal states of a planning stage that performs motion prediction based on common behavior patterns. The entire system is modeled as one monolithic neural network and trained via inverse reinforcement learning. Experimental validation on real world data shows the system's ability to predict both, destinations and trajectories accurately

A Data-driven Model for Interaction-aware Pedestrian Motion Prediction in Object Cluttered Environments

This paper reports on a data-driven, interaction-aware motion prediction approach for pedestrians in environments cluttered with static obstacles. When navigating in such workspaces shared with humans, robots need accurate motion predictions of the surrounding pedestrians. Human navigation behavior is mostly influenced by their surrounding pedestrians and by the static obstacles in their vicinity. In this paper we introduce a new model based on Long-Short Term Memory (LSTM) neural networks, which is able to learn human motion behavior from demonstrated data. To the best of our knowledge, this is the first approach using LSTMs, that incorporates both static obstacles and surrounding pedestrians for trajectory forecasting. As part of the model, we introduce a new way of encoding surrounding pedestrians based on a 1d-grid in polar angle space. We evaluate the benefit of interaction-aware motion prediction and the added value of incorporating static obstacles on both simulation and real-world datasets by comparing with state-of-the-art approaches. The results show, that our new approach outperforms the other approaches while being very computationally efficient and that taking into account static obstacles for motion predictions significantly improves the prediction accuracy, especially in cluttered environments.Comment: 8 pages, accepted for publication at the IEEE International Conference on Robotics and Automation (ICRA) 201

A Data-driven Model for Interaction-aware Pedestrian Motion Prediction in Object Cluttered Environments

This paper reports on a data-driven, interaction-aware motion prediction approach for pedestrians in environments cluttered with static obstacles. When navigating in such workspaces shared with humans, robots need accurate motion predictions of the surrounding pedestrians. Human navigation behavior is mostly influenced by their surrounding pedestrians and by the static obstacles in their vicinity. In this paper we introduce a new model based on Long-Short Term Memory (LSTM) neural networks, which is able to learn human motion behavior from demonstrated data. To the best of our knowledge, this is the first approach using LSTMs, that incorporates both static obstacles and surrounding pedestrians for trajectory forecasting. As part of the model, we introduce a new way of encoding surrounding pedestrians based on a 1d-grid in polar angle space. We evaluate the benefit of interaction-aware motion prediction and the added value of incorporating static obstacles on both simulation and real-world datasets by comparing with state-of-the-art approaches. The results show, that our new approach outperforms the other approaches while being very computationally efficient and that taking into account static obstacles for motion predictions significantly improves the prediction accuracy, especially in cluttered environments.Comment: 8 pages, accepted for publication at the IEEE International Conference on Robotics and Automation (ICRA) 201

보행자 거동 및 운전자 주행 특성 기반의 자율주행 종방향 거동 계획

학위논문 (석사) -- 서울대학교 대학원 : 공과대학 기계공학부, 2020. 8. 이경수.본 연구는 보행자의 미래 거동 방향에 대한 불확실성을 고려한 보행자 모델을 제안하고, 보행자 대응 시의 운전자 주행 특성을 반영하여 자율주행 차량의 종방향 모션을 계획하는 알고리즘을 제시한다. 도심 자율 주행을 가능하게 하기위해서는 보행자와의 상호적인 주행이 필수적이다. 그러나, 보행자는 거동 방향 전환이 쉽게 일어나기 때문에 미래 거동을 예측하기가 어렵고, 이에 대응하는 자차의 거동을 결정짓는 데도 어려움이 있다. 이러한 보행자의 거동 불확실성이 존재함에도 자율 주행 차량이 보행자의 안전성을 확보하고 휴먼 운전자와 같이 거동하기 위해서는, 보행자의 거동 불확실성을 반영하는 보행자 모델이 우선적으로 필요하다. 해당 연구에서는 보행자 거동 특성을 조사하여 보행자 거동 확률 모델을 정의하고, 보행자 대응 상황에서의 운전자의 거동을 조사하여 자율주행 차량의 종방향 거동 계획에 적용한다. 해당 논문은 크게 보행자 모델 정의, 예측 기반 충돌 위험 평가 그리고 보행자 대응 종방향 거동 계획의 세 가지 주요 파트로 이루어져 있다. 첫 번째 파트에서 보행자 모델 정의의 핵심 이론은 보행자의 거동 속도와 방향을 전환하는 거동 사이에는 특정 상관관계를 가지고 있다는 것이다. 보행자의 거동 특성은 자율 주행 차량에 부착된 라이다 센서와 전방 카메라를 통해 획득한 보행자 데이터를 통계적으로 분석한 결과로 도출되었다. 해당 데이터를 통해 속도에 따라 보행자가 모든 방향에 대해서 거동할 확률이 도출되고, 보행자의 미래 거동 범위는 도출된 확률 분포에서 유효 시그마 범위를 설정하여 구획된다. 이는 보행자가 일정 시간 동안 특정 확률로 거동할 영역을 고려하여, 위험이 존재할 수 있는 보행자에 대해서 미리 차량의 움직임을 계획할 수 있도록 한다. 두 번째 파트로 보행자와 자 차량의 일정 시간 동안의 위치 정보를 예측하여 충돌 위험성을 평가한다. 보행자 예측은 앞서 도출한 보행자 유효 예측 거동 범위 내에서 가장 위험성이 큰 방향으로 움직인다고 가정한다. 또한, 자 차량의 경우 주어진 로컬 경로를 따라 움직인다는 가정을 하는 차선 유지 모델을 사용한다. 예측 결과를 통해 현재 추가적인 감속도를 가하지 않았을 때, 충돌 위험이 존재하는지 확인한다. 마지막으로, 타겟이 되는 보행자에 대한 종방향 거동을 결정한다. 우선적으로 보행자 대응 상황에서 적절한 감속도와 감속 시점을 결정하기 위해 휴먼 운전자 주행 데이터를 분석한다. 이를 통해 주행에서 핵심적인 파라미터들이 정의되고, 해당 파라미터들은 종방향 거동 계획에 반영된다. 따라서 최종적으로 보행자 예측 거동 영역에 대해서 자율 주행 차량의 추종 가속도이 결정된다. 제시된 알고리즘은 실차 테스트를 통해 성능이 확인된다. 테스트 결과, 도출한 보행자 모델과 예측 모델을 바탕으로 한 감속 결정 시점과 감속도의 궤적이 동일 상황들에 대해서 능숙한 운전자와 유사함이 확인되었다.This paper presents a pedestrian model considering uncertainty in the direction of future movement and a human-like longitudinal motion planning algorithm for autonomous vehicle in the interaction situation with pedestrians. Interactive driving with pedestrians is essential for autonomous driving in urban environments. However, interaction with pedestrians is very challenging for autonomous vehicle because it is difficult to predict movement direction of pedestrians. Even if there exists uncertainty of the behavior of pedestrians, the autonomous vehicles should plan their motions ensuring pedestrian safety and respond smoothly to pedestrians. To implement this, a pedestrian probabilistic yaw model is introduced based on behavioral characteristics and the human driving parameters are investigated in the interaction situation. The paper consists of three main parts: the pedestrian model definition, collision risk assessment based on prediction and human-like longitudinal motion planning. In the first section, the main key of pedestrian model is the behavior tendency with correlation between pedestrians speed and direction change. The behavior characteristics are statistically investigated based on perceived pedestrian tracking data using light detection and ranging(Lidar) sensor and front camera. Through the behavior characteristics, movement probability for all directions of the pedestrian is derived according to pedestrians velocity. Also, the effective moving area can be limited up to the valid probability criterion. The defined model allows the autonomous vehicle to know the area that pedestrian may head to a certain probability in the future steps. This helps to plan the vehicle motion considering the pedestrian yaw states uncertainty and to predetermine the motion of autonomous vehicle from the pedestrians who may have a risk. Secondly, a risk assessment is required and is based on the pedestrian model. The dynamic states of pedestrians and subject vehicle are predicted to do a risk assessment. In this section, the pedestrian behavior is predicted under the assumption of moving to the most dangerous direction in the effective moving area obtained above. The prediction of vehicle behavior is performed using a lane keeping model in which the vehicle follows a given path. Based on the prediction result, it is checked whether there will be a collision between the pedestrian and the vehicle if deceleration motion is not taken. Finally, longitudinal motion planning is determined for target pedestrians with possibility of collision. Human driving data is first examined to obtain a proper longitudinal deceleration and deceleration starting point in the interaction situation with pedestrians. Several human driving parameters are defined and applied in determining the longitudinal acceleration of the vehicle. The longitudinal motion planning algorithm is verified via vehicle tests. The test results confirm that the proposed algorithm shows similar longitudinal motion and deceleration decision to a human driver based on predicted pedestrian model.Chapter 1. Introduction 1 1.1. Background and Motivation 1 1.2. Previous Researches 3 1.3. Thesis Objective and Outline 5 Chapter 2. Probabilistic Pedestrian Yaw Model 8 2.1. Pedestrian Behavior Characteristics 9 2.2. Probability Movement Range 11 Chapter 3. Prediction Based Risk Assessment 13 3.1. Lane Keeping Behavior Model 15 3.2. Subject Vehicle Prediction 17 3.3. Safety Region Based on Prediction 19 Chapter 4. Human-like Longitudinal Motion Planning 22 4.1. Human Driving Parameters Definition 22 4.1.1 Hard Mode Distance 23 4.1.2 Soft Mode Distance and Velocity 23 4.1.3 Time-To-Collision 23 4.2. Driving Mode and Acceleration Decision 25 4.2.1 Acceleration of Each Mode 25 4.2.2 Mode Selection 26 Chapter 5. Vehicle Test Result 28 5.1. Configuration of Experimental Vehicle 28 5.2. Longitudinal Motion Planning for Pedestiran 30 5.2.1 Soft Mode Scenario 32 5.2.2 Hard Mode Scenario 35 Chapter 6. Colclusion 38 Bibliography 39 국문 초록 42Maste

Modeling Cooperative Navigation in Dense Human Crowds

For robots to be a part of our daily life, they need to be able to navigate among crowds not only safely but also in a socially compliant fashion. This is a challenging problem because humans tend to navigate by implicitly cooperating with one another to avoid collisions, while heading toward their respective destinations. Previous approaches have used hand-crafted functions based on proximity to model human-human and human-robot interactions. However, these approaches can only model simple interactions and fail to generalize for complex crowded settings. In this paper, we develop an approach that models the joint distribution over future trajectories of all interacting agents in the crowd, through a local interaction model that we train using real human trajectory data. The interaction model infers the velocity of each agent based on the spatial orientation of other agents in his vicinity. During prediction, our approach infers the goal of the agent from its past trajectory and uses the learned model to predict its future trajectory. We demonstrate the performance of our method against a state-of-the-art approach on a public dataset and show that our model outperforms when predicting future trajectories for longer horizons.Comment: Accepted at ICRA 201

Human Motion Trajectory Prediction: A Survey

With growing numbers of intelligent autonomous systems in human environments, the ability of such systems to perceive, understand and anticipate human behavior becomes increasingly important. Specifically, predicting future positions of dynamic agents and planning considering such predictions are key tasks for self-driving vehicles, service robots and advanced surveillance systems. This paper provides a survey of human motion trajectory prediction. We review, analyze and structure a large selection of work from different communities and propose a taxonomy that categorizes existing methods based on the motion modeling approach and level of contextual information used. We provide an overview of the existing datasets and performance metrics. We discuss limitations of the state of the art and outline directions for further research.Comment: Submitted to the International Journal of Robotics Research (IJRR), 37 page