Slanted Stixels: A way to represent steep streets

This work presents and evaluates a novel compact scene representation based on Stixels that infers geometric and semantic information. Our approach overcomes the previous rather restrictive geometric assumptions for Stixels by introducing a novel depth model to account for non-flat roads and slanted objects. Both semantic and depth cues are used jointly to infer the scene representation in a sound global energy minimization formulation. Furthermore, a novel approximation scheme is introduced in order to significantly reduce the computational complexity of the Stixel algorithm, and then achieve real-time computation capabilities. The idea is to first perform an over-segmentation of the image, discarding the unlikely Stixel cuts, and apply the algorithm only on the remaining Stixel cuts. This work presents a novel over-segmentation strategy based on a Fully Convolutional Network (FCN), which outperforms an approach based on using local extrema of the disparity map. We evaluate the proposed methods in terms of semantic and geometric accuracy as well as run-time on four publicly available benchmark datasets. Our approach maintains accuracy on flat road scene datasets while improving substantially on a novel non-flat road dataset.Comment: Journal preprint (published in IJCV 2019: https://link.springer.com/article/10.1007/s11263-019-01226-9). arXiv admin note: text overlap with arXiv:1707.0539

Exploring the Range of Possible Outcomes by means of Logical Scenario Analysis and Reduction for Testing Automated Driving Systems

With the implementation of the new EU regulation 2022/1426 regarding the type-approval of the automated driving system (ADS) of fully automated vehicles, scenario-based testing has gained significant importance in evaluating the performance and safety of advanced driver assistance systems and automated driving systems. However, the exploration and generation of concrete scenarios from a single logical scenario can often lead to a number of similar or redundant scenarios, which may not contribute to the testing goals. This paper focuses on the the goal to reduce the scenario set by clustering concrete scenarios from a single logical scenario. By employing clustering techniques, redundant and uninteresting scenarios can be identified and eliminated, resulting in a representative scenario set. This reduction allows for a more focused and efficient testing process, enabling the allocation of resources to the most relevant and critical scenarios. Furthermore, the identified clusters can provide valuable insights into the scenario space, revealing patterns and potential problems with the system's behavior.Comment: submitted to IEEE ITSC 202

Point Cloud Processing Algorithms for Environment Understanding in Intelligent Vehicle Applications

Understanding the surrounding environment including both still and moving objects is crucial to the design and optimization of intelligent vehicles. In particular, acquiring the knowledge about the vehicle environment could facilitate reliable detection of moving objects for the purpose of avoiding collisions. In this thesis, we focus on developing point cloud processing algorithms to support intelligent vehicle applications. The contributions of this thesis are three-fold.;First, inspired by the analogy between point cloud and video data, we propose to formulate a problem of reconstructing the vehicle environment (e.g., terrains and buildings) from a sequence of point cloud sets. Built upon existing point cloud registration tool such as iterated closest point (ICP), we have developed an expectation-maximization (EM)-like technique that can automatically mosaic multiple point cloud sets into a larger one characterizing the still environment surrounding the vehicle.;Second, we propose to utilize the color information (from color images captured by the RGB camera) as a supplementary source to the three-dimensional point cloud data. Such joint color and depth representation has the potential of better characterizing the surrounding environment of a vehicle. Based on the novel joint RGBD representation, we propose training a convolution neural network on color images and depth maps generated from the point cloud data.;Finally, we explore a sensor fusion method that combines the results given by a Lidar based detection algorithm and vehicle to everything (V2X) communicated data. Since Lidar and V2X respectively characterize the environmental information from complementary sources, we propose to get a better localization of the surrounding vehicles by a linear sensor fusion method. The effectiveness of the proposed sensor fusion method is verified by comparing detection error profiles

Gen-LaneNet: A Generalized and Scalable Approach for 3D Lane Detection

We present a generalized and scalable method, called Gen-LaneNet, to detect 3D lanes from a single image. The method, inspired by the latest state-of-the-art 3D-LaneNet, is a unified framework solving image encoding, spatial transform of features and 3D lane prediction in a single network. However, we propose unique designs for Gen-LaneNet in two folds. First, we introduce a new geometry-guided lane anchor representation in a new coordinate frame and apply a specific geometric transformation to directly calculate real 3D lane points from the network output. We demonstrate that aligning the lane points with the underlying top-view features in the new coordinate frame is critical towards a generalized method in handling unfamiliar scenes. Second, we present a scalable two-stage framework that decouples the learning of image segmentation subnetwork and geometry encoding subnetwork. Compared to 3D-LaneNet, the proposed Gen-LaneNet drastically reduces the amount of 3D lane labels required to achieve a robust solution in real-world application. Moreover, we release a new synthetic dataset and its construction strategy to encourage the development and evaluation of 3D lane detection methods. In experiments, we conduct extensive ablation study to substantiate the proposed Gen-LaneNet significantly outperforms 3D-LaneNet in average precision(AP) and F-score

CoDR: Correlation-based Data Reduction Scheme for Efficient Gathering of Heterogeneous Driving Data

A variety of deep learning techniques are actively employed for advanced driver assistance systems, which in turn require gathering lots of heterogeneous driving data, such as traffic conditions, driver behavior, vehicle status and location information. However, these different types of driving data easily become more than tens of GB per day, forming a significant hurdle due to the storage and network cost. To address this problem, this paper proposes a novel scheme, called CoDR, which can reduce data volume by considering the correlations among heterogeneous driving data. Among heterogeneous datasets, CoDR first chooses one set as a pivot data. Then, according to the objective of data collection, it identifies data ranges relevant to the objective from the pivot dataset. Finally, it investigates correlations among sets, and reduces data volume by eliminating irrelevant data from not only the pivot set but also other remaining datasets. CoDR gathers four heterogeneous driving datasets: two videos for front view and driver behavior, OBD-II and GPS data. We show that CoDR decreases data volume by up to 91%. We also present diverse analytical results that reveal the correlations among the four datasets, which can be exploited usefully for edge computing to reduce data volume on the spot. Document type: Articl

A Strain-Based Method to Detect Tires' Loss of Grip and Estimate Lateral Friction Coefficient from Experimental Data by Fuzzy Logic for Intelligent Tire Development

This article belongs to the Special Issue State-of-the-Art Sensors Technology in Spain 2017Tires are a key sub-system of vehicles that have a big responsibility for comfort, fuel consumption and traffic safety. However, current tires are just passive rubber elements which do not contribute actively to improve the driving experience or vehicle safety. The lack of information from the tire during driving gives cause for developing an intelligent tire. Therefore, the aim of the intelligent tire is to monitor tire working conditions in real-time, providing useful information to other systems and becoming an active system. In this paper, tire tread deformation is measured to provide a strong experimental base with different experiments and test results by means of a tire fitted with sensors. Tests under different working conditions such as vertical load or slip angle have been carried out with an indoor tire test rig. The experimental data analysis shows the strong relation that exists between lateral force and the maximum tensile and compressive strain peaks when the tire is not working at the limit of grip. In the last section, an estimation system from experimental data has been developed and implemented in Simulink to show the potential of strain sensors for developing intelligent tire systems, obtaining as major results a signal to detect tire's loss of grip and estimations of the lateral friction coefficient

Interaction-Aware Motion Planning for Automated Vehicles

Die Bewegungsplanung für automatisierte Fahrzeuge (AVs) in gemischtem Verkehr ist eine herausfordernde Aufgabe. Hierbei bezeichnet gemischter Verkehr, Verkehr bestehend aus von Menschen gefahrenen Fahrzeugen sowie automatisierten Fahrzeugen. Um die Komplexität der Aufgabe zu reduzieren, verwenden state-of-the-art Planungsansätze oft die vereinfachende Annahme, dass das zukünftige Verhalten umliegender Fahrzeuge unabhängig vom Plan des AVs vorhergesagt werden kann. Während die Trennung von Prädiktion und Planung für viele Verkehrssituationen eine hilfreiche Vereinfachung darstellt, werden hierbei Interaktionen zwischen den Verkehrsteilnehmern ignoriert, was besonders in interaktiven Verkehrssituationen zu suboptimalem, übermäßig konservativem Fahrverhalten führen kann. In dieser Arbeit werden zwei interaktionsbewusste Bewegungsplanungsalgorithmen vorgeschlagen, die in der Lage sind übermäßig konservatives Fahrverhalten zu reduzieren. Der Kernaspekt dieser Algorithmen ist, dass Prädiktion und Planung gleichzeitig gelöst werden. Mit diesen Algorithmen können anspruchsvolle Fahrmanöver, wie z. B. das Reißverschlussverfahren in dichtem Verkehr, durchgeführt werden, die mit state-of-the-art Planungsansätzen nicht möglich sind. Der erste Algorithmus basiert auf Methoden der Multi-Agenten-Planung. Interaktionen zwischen Verkehrsteilnehmern werden durch Optimierung gekoppelter Trajektorien mittels einer gemeinsamen Kostenfunktion approximiert. Das Kernstück des Algorithmus ist eine neuartige Multi-Agenten-Trajektorienplanungsformulierung, die auf gemischt-ganzzahliger quadratischer Programmierung (MIQP) basiert. Die Formulierung garantiert global optimale Lösungen und ist somit in der Lage das kombinatorische Problem zu lösen, welches kontinuierliche Methoden auf lokal optimale Lösungen beschränkt. Desweiteren kann durch den vorgestellten Ansatz ein manöverneutrales Verhalten erzeugt werden, das Manöverentscheidungen in ungewissen Situationen aufschieben kann. Der zweite Ansatz formuliert Interaktionen zwischen einem menschlichen Fahrer und einem AV als ein Stackelberg-Spiel. Im Gegensatz zu bestehenden Arbeiten kann der Algorithmus allgemeine nichtlineare Zustands- und Eingabebeschränkungen berücksichtigen. Desweiteren führen wir Mechanismen zur Integration von Kooperation und Rücksichtnahme in die Planung ein. Damit wird übermäßig aggressives Fahrverhalten verhindert, was in der Literatur als ein Problem interaktionsbewusster Planungsmethoden identifiziert wurde. Die Wirksamkeit, Robustheit und Echtzeitfähigkeit des Algorithmus wird durch numerische Experimente gezeigt

Pedestrian and cyclist detection and intent estimation for autonomous vehicles: A survey

© 2019 by the authors. As autonomous vehicles become more common on the roads, their advancement draws on safety concerns for vulnerable road users, such as pedestrians and cyclists. This paper presents a review of recent developments in pedestrian and cyclist detection and intent estimation to increase the safety of autonomous vehicles, for both the driver and other road users. Understanding the intentions of the pedestrian/cyclist enables the self-driving vehicle to take actions to avoid incidents. To make this possible, development of methods/techniques, such as deep learning (DL), for the autonomous vehicle will be explored. For example, the development of pedestrian detection has been significantly advanced using DL approaches, such as; Fast Region-Convolutional Neural Network (R-CNN), Faster R-CNN and Single Shot Detector (SSD). Although DL has been around for several decades, the hardware to realise the techniques have only recently become viable. Using these DL methods for pedestrian and cyclist detection and applying it for the tracking, motion modelling and pose estimation can allow for a successful and accurate method of intent estimation for the vulnerable road users. Although there has been a growth in research surrounding the study of pedestrian detection using vision-based approaches, further attention should include focus on cyclist detection. To further improve safety for these vulnerable road users (VRUs), approaches such as sensor fusion and intent estimation should be investigated