RANSAC for Robotic Applications: A Survey

Random Sample Consensus, most commonly abbreviated as RANSAC, is a robust estimation method for the parameters of a model contaminated by a sizable percentage of outliers. In its simplest form, the process starts with a sampling of the minimum data needed to perform an estimation, followed by an evaluation of its adequacy, and further repetitions of this process until some stopping criterion is met. Multiple variants have been proposed in which this workflow is modified, typically tweaking one or several of these steps for improvements in computing time or the quality of the estimation of the parameters. RANSAC is widely applied in the field of robotics, for example, for finding geometric shapes (planes, cylinders, spheres, etc.) in cloud points or for estimating the best transformation between different camera views. In this paper, we present a review of the current state of the art of RANSAC family methods with a special interest in applications in robotics.This work has been partially funded by the Basque Government, Spain, under Research Teams Grant number IT1427-22 and under ELKARTEK LANVERSO Grant number KK-2022/00065; the Spanish Ministry of Science (MCIU), the State Research Agency (AEI), the European Regional Development Fund (FEDER), under Grant number PID2021-122402OB-C21 (MCIU/AEI/FEDER, UE); and the Spanish Ministry of Science, Innovation and Universities, under Grant FPU18/04737

3D Modelling for Improved Visual Traffic Analytics

Advanced Traffic Management Systems utilize diverse types of sensor networks with the goal of improving mobility and safety of transportation systems. These systems require information about the state of the traffic configuration, including volume, vehicle speed, density, and incidents, which are useful in applications such as urban planning, collision avoidance systems, and emergency vehicle notification systems, to name a few. Sensing technologies are an important part of Advanced Traffic Management Systems that enable the estimation of the traffic state. Inductive Loop Detectors are often used to sense vehicles on highway roads. Although this technology has proven to be effective, it has limitations. Their installation and replacement cost is high and causes traffic disruptions, and their sensing modality provides very limited information about the vehicles being sensed. No vehicle appearance information is available. Traffic camera networks are also used in advanced traffic monitoring centers where the cameras are controlled by a remote operator. The amount of visual information provided by such cameras can be overwhelmingly large, which may cause the operators to miss important traffic events happening in the field. This dissertation focuses on visual traffic surveillance for Advanced Traffic Management Systems. The focus is on the research and development of computer vision algorithms that contribute to the automation of highway traffic analytics systems that require estimates of traffic volume and density. This dissertation makes three contributions: The first contribution is an integrated vision surveillance system called 3DTown, where cameras installed at a university campus together with algorithms are used to produce vehicle and pedestrian detections to augment a 3D model of the university with dynamic information from the scene. A second major contribution is a technique for extracting road lines from highway images that are used to estimate the tilt angle and the focal length of the camera. This technique is useful when the operator changes the camera pose. The third major contribution is a method to automatically extract the active road lanes and model the vehicles in 3D to improve the vehicle count estimation by individuating 2D segments of imaged vehicles that have been merged due to occlusions

Context Exploitation in Data Fusion

Complex and dynamic environments constitute a challenge for existing tracking algorithms. For this reason, modern solutions are trying to utilize any available information which could help to constrain, improve or explain the measurements. So called Context Information (CI) is understood as information that surrounds an element of interest, whose knowledge may help understanding the (estimated) situation and also in reacting to that situation. However, context discovery and exploitation are still largely unexplored research topics. Until now, the context has been extensively exploited as a parameter in system and measurement models which led to the development of numerous approaches for the linear or non-linear constrained estimation and target tracking. More specifically, the spatial or static context is the most common source of the ambient information, i.e. features, utilized for recursive enhancement of the state variables either in the prediction or the measurement update of the filters. In the case of multiple model estimators, context can not only be related to the state but also to a certain mode of the filter. Common practice for multiple model scenarios is to represent states and context as a joint distribution of Gaussian mixtures. These approaches are commonly referred as the join tracking and classification. Alternatively, the usefulness of context was also demonstrated in aiding the measurement data association. Process of formulating a hypothesis, which assigns a particular measurement to the track, is traditionally governed by the empirical knowledge of the noise characteristics of sensors and operating environment, i.e. probability of detection, false alarm, clutter noise, which can be further enhanced by conditioning on context. We believe that interactions between the environment and the object could be classified into actions, activities and intents, and formed into structured graphs with contextual links translated into arcs. By learning the environment model we will be able to make prediction on the target\u2019s future actions based on its past observation. Probability of target future action could be utilized in the fusion process to adjust tracker confidence on measurements. By incorporating contextual knowledge of the environment, in the form of a likelihood function, in the filter measurement update step, we have been able to reduce uncertainties of the tracking solution and improve the consistency of the track. The promising results demonstrate that the fusion of CI brings a significant performance improvement in comparison to the regular tracking approaches

Driver lane change intention inference using machine learning methods.

Lane changing manoeuvre on highway is a highly interactive task for human drivers. The intelligent vehicles and the advanced driver assistance systems (ADAS) need to have proper awareness of the traffic context as well as the driver. The ADAS also need to understand the driver potential intent correctly since it shares the control authority with the human driver. This study provides a research on the driver intention inference, particular focus on the lane change manoeuvre on highways. This report is organised in a paper basis, where each chapter corresponding to a publication, which is submitted or to be submitted. Part Ⅰ introduce the motivation and general methodology framework for this thesis. Part Ⅱ includes the literature survey and the state-of-art of driver intention inference. Part Ⅲ contains the techniques for traffic context perception that focus on the lane detection. A literature review on lane detection techniques and its integration with parallel driving framework is proposed. Next, a novel integrated lane detection system is designed. Part Ⅳ contains two parts, which provides the driver behaviour monitoring system for normal driving and secondary tasks detection. The first part is based on the conventional feature selection methods while the second part introduces an end-to-end deep learning framework. The design and analysis of driver lane change intention inference system for the lane change manoeuvre is proposed in Part Ⅴ. Finally, discussions and conclusions are made in Part Ⅵ. A major contribution of this project is to propose novel algorithms which accurately model the driver intention inference process. Lane change intention will be recognised based on machine learning (ML) methods due to its good reasoning and generalizing characteristics. Sensors in the vehicle are used to capture context traffic information, vehicle dynamics, and driver behaviours information. Machine learning and image processing are the techniques to recognise human driver behaviour.PhD in Transpor

Short-Term Traffic Participants Behaviour Prediction for Automated Vehicles

Due to the rapid commencement of autonomous vehicles and the promising potential benefits, it has made it critical for said vehicles to be able to interpret their environment and compensate for the absence of driver predictions from visual cues. This study presents a novel intermediate component to improve the performance of autonomous vehicle controllers, by providing them with real-time microscopic predictions of traffic participants' behaviour, given the environmental conditions. This strategy is especially aimed towards direct combination with model-predictive controllers (MPCs) and other controllers that can utilize dynamic state predictions. This task is undertaken in three stages for three different scenarios. Scenario I considers V2X communications and predicts the velocity of an arbitrary vehicle in longitudinal direction. Using a recurrent neural network (RNN) and considering a complementary variable the strategy can predict the speed profile of said vehicle for arbitrary horizons. Results of this scenario exhibit >0.95 correlation if trained with enough data. Scenario II moves on to a more sophisticated approach for prediction of vehicles on US-101, using real data provided by the U.S. Federal Highway Administration (FHWA) under NGSIM. Utilizing a marriage of dynamic Bayesian network (DBN) and RNN, the method can make predictions on speed profiles of all present vehicles within a range, for arbitrary horizons, as well as prediction on whether the vehicle on the main lanes would yield to the merging vehicles on the ramp. Due to digital nature of the DBN stream, a Kalman filter (KF) was introduced as post processing smoothing method. Results of this scenario exhibit >0.95 correlation and <1.6 mph mean absolute error. Scenario III tackles a much more complex driving environment, intersection driving. Because in intersection driving, the priority relationships of highway driving are no longer existent, the training must be broadened to encompass vehicle pairs which is exponentially more difficult than training for single vehicles. The data for this phase was generated by SUMO. Results of this scenario exhibit <1.1 mph mean absolute error. Scenario IV focuses on the problem of roundabout driving. In roundabout driving, the general driving situation is more similar to highway merging, however due to the rapid move toward replacing intersections with roundabouts, especially in developing cities, definitely an important scenario to look at. In this scenario SUMO was used for data generation, a new DBN topology was developed and the results yielded exhibit >0.89 correlation. To evaluate the performance and the accuracy of the proposed method, it was compared with a collection of sequence prediction techniques, including LSTM and GRU. It was concluded that the DBN-RNN has the best accuracy and performance among these methods. Validation of the strategy was planned to be done on the scaled autonomous vehicle test platform developed in Smart Hybrid and Electric Vehicles Systems (SHEVS) lab, where driver-in-the-loop hardware was incorporated and the equipment were prepared but due to COVID-19 closures was not realized

Probabilistic traffic breakdown forecasting through Bayesian approximation using variational LSTMs

Robust artificial intelligence models have been criticized for their lack of uncertainty control and inability to explain feature importance, which has limited their adoption. However, probabilistic machine learning and explainable artificial intelligence have shown great scientific and technical advances, and have slowly permeated other areas, such as Traffic Engineering. This thesis fulfils a literature gap related to probabilistic traffic breakdown forecasting. We propose a traffic breakdown probability calculation methodology based on probabilistic speed predictions. Since the probabilistic characteristic is absent in traditional formulations of neural networks, we suggest using Variational LSTMs to make the speed forecasts. This Recurrent Neural Network uses Dropout to produce a Bayesian approximation and generate probabilistic outputs. This thesis also investigates the effects of inclement weather on traffic breakdown probability and methods for identifying traffic breakdowns. The proposed methodology produces great control over the probability of congestion, which could not be achieved using deterministic models, resulting in important theoretical and practical contributions

Predictive Maneuver Planning and Control of an Autonomous Vehicle in Multi-Vehicle Traffic with Observation Uncertainty

Autonomous vehicle technology is a promising development for improving the safety, efficiency and environmental impact of on-road transportation systems. However, the task of guiding an autonomous vehicle by rapidly and systematically accommodating the plethora of changing constraints, e.g. of avoiding multiple stationary and moving obstacles, obeying traffic rules, signals and so on as well as the uncertain state observation due to sensor imperfections, remains a major challenge. This dissertation attempts to address this challenge via designing a robust and efficient predictive motion planning framework that can generate the appropriate vehicle maneuvers (selecting and tracking specific lanes, and related speed references) as well as the constituent motion trajectories while considering the differential vehicle kinematics of the controlled vehicle and other constraints of operating in public traffic. The main framework combines a finite state machine (FSM)-based maneuver decision module with a model predictive control (MPC)-based trajectory planner. Based on the prediction of the traffic environment, reference speeds are assigned to each lane in accordance with the detection of objects during measurement update. The lane selection decisions themselves are then incorporated within the MPC optimization. The on-line maneuver/motion planning effort for autonomous vehicles in public traffic is a non-convex problem due to the multiple collision avoidance constraints with overlapping areas, lane boundaries, and nonlinear vehicle-road dynamics constraints. This dissertation proposes and derives some remedies for these challenges within the planning framework to improve the feasibility and optimality of the solution. Specifically, it introduces vehicle grouping notions and derives conservative and smooth algebraic models to describe the overlapped space of several individual infeasible spaces and help prevent the optimization from falling into undesired local minima. Furthermore, in certain situations, a forced objective selection strategy is needed and adopted to help the optimization jump out of local minima. Furthermore, the dissertation considers stochastic uncertainties prevalent in dynamic and complex traffic and incorporate them with in the predictive planning and control framework. To this end, Bayesian filters are implemented to estimate the uncertainties in object motions and then propagate them into the prediction horizon. Then, a pair-wise probabilistic collision condition is defined for objects with non-negligible geometrical shape/sizes and computationally efficient and conservative forms are derived to efficiently and analytically approximate the involved multi-variate integrals. The probabilistic collision evaluation is then applied within a vehicle grouping algorithms to cluster the object vehicles with closeness in positions and speeds and eventually within the stochastic predictive maneuver planner framework to tighten the chanced-constraints given a deterministic confidence margin. It is argued that these steps make the planning problem tractable for real-time implementation on autonomously controlled vehicles