Combining LiDAR Space Clustering and Convolutional Neural Networks for Pedestrian Detection

Pedestrian detection is an important component for safety of autonomous vehicles, as well as for traffic and street surveillance. There are extensive benchmarks on this topic and it has been shown to be a challenging problem when applied on real use-case scenarios. In purely image-based pedestrian detection approaches, the state-of-the-art results have been achieved with convolutional neural networks (CNN) and surprisingly few detection frameworks have been built upon multi-cue approaches. In this work, we develop a new pedestrian detector for autonomous vehicles that exploits LiDAR data, in addition to visual information. In the proposed approach, LiDAR data is utilized to generate region proposals by processing the three dimensional point cloud that it provides. These candidate regions are then further processed by a state-of-the-art CNN classifier that we have fine-tuned for pedestrian detection. We have extensively evaluated the proposed detection process on the KITTI dataset. The experimental results show that the proposed LiDAR space clustering approach provides a very efficient way of generating region proposals leading to higher recall rates and fewer misses for pedestrian detection. This indicates that LiDAR data can provide auxiliary information for CNN-based approaches

Object Detection Using LiDAR and Camera Fusion in Off-road Conditions

Seoses hüppelise huvi kasvuga autonoomsete sõidukite vastu viimastel aastatel on suurenenud ka vajadus täpsemate ja töökindlamate objektituvastuse meetodite järele. Kuigi tänu konvolutsioonilistele närvivõrkudele on palju edu saavutatud 2D objektituvastuses, siis võrreldavate tulemuste saavutamine 3D maailmas on seni jäänud unistuseks. Põhjuseks on mitmesugused probleemid eri modaalsusega sensorite andmevoogude ühitamisel, samuti on 3D maailmas märgendatud andmestike loomine aeganõudvam ja kallim. Sõltumata sellest, kas kasutame objektide kauguse hindamiseks stereo kaamerat või lidarit, kaasnevad andmevoogude ühitamisega ajastusprobleemid, mis raskendavad selliste lahenduste kasutamist reaalajas. Lisaks on enamus olemasolevaid lahendusi eelkõige välja töötatud ja testitud linnakeskkonnas liikumiseks.Töös pakutakse välja meetod 3D objektituvastuseks, mis põhineb 2D objektituvastuse tulemuste (objekte ümbritsevad kastid või segmenteerimise maskid) projitseerimisel 3D punktipilve ning saadud punktipilve filtreerimisel klasterdamismeetoditega. Tulemusi võrreldakse lihtsa termokaamera piltide filtreerimisel põhineva lahendusega. Täiendavalt viiakse läbi põhjalikud eksperimendid parimate algoritmi parameetrite leidmiseks objektituvastuseks maastikul, saavutamaks suurimat võimalikku täpsust reaalajas.Since the boom in the industry of autonomous vehicles, the need for preciseenvironment perception and robust object detection methods has grown. While we are making progress with state-of-the-art in 2D object detection with approaches such as convolutional neural networks, the challenge remains in efficiently achieving the same level of performance in 3D. The reasons for this include limitations of fusing multi-modal data and the cost of labelling different modalities for training such networks. Whether we use a stereo camera to perceive scene’s ranging information or use time of flight ranging sensors such as LiDAR, ​ the existing pipelines for object detection in point clouds have certain bottlenecks and latency issues which tend to affect the accuracy of detection in real time speed. Moreover, ​ these existing methods are primarily implemented and tested over urban cityscapes.This thesis presents a fusion based approach for detecting objects in 3D by projecting the proposed 2D regions of interest (object’s bounding boxes) or masks (semantically segmented images) to point clouds and applies outlier filtering techniques to filter out target object points in projected regions of interest. Additionally, we compare it with human detection using thermal image thresholding and filtering. Lastly, we performed rigorous benchmarks over the off-road environments to identify potential bottlenecks and to find a combination of pipeline parameters that can maximize the accuracy and performance of real-time object detection in 3D point clouds

Multi-Sensor Fusion for 3D Object Detection

Sensing and modelling of the surrounding environment is crucial for solving many of the problems in intelligent machines like self-driving cars, autonomous robots, and augmented reality displays. Performance, reliability and safety of the autonomous agents rely heavily on the way the environment is modelled. Two-dimensional models are inadequate to capture the three-dimensional nature of real-world scenes. Three-dimensional models are necessary to achieve the standards required by the autonomy stack for intelligent agents to work alongside humans. Data driven deep learning methodologies for three-dimensional scene modelling has evolved greatly in the past few years because of the availability of huge amounts of data from variety of sensors in the form of well-designed datasets. 3D object detection and localization are two of the key requirements for tasks such as obstacle avoidance, agent-to-agent interaction, and path planning. Most methodologies for object detection work on a single sensor data like camera or LiDAR. Camera sensors provide feature rich scene data and LiDAR provides us 3D geometrical information. Advanced object detection and localization can be achieved by leveraging the information from both camera and LiDAR sensors. In order to effectively quantify the uncertainty of each sensor channel, an appropriate fusion strategy is needed to fuse the independently encoded point clouds from LiDAR with the RGB images from standard vision cameras. In this work, we introduce a fusion strategy and develop a multimodal pipeline which utilizes existing state-of-the-art deep learning based data encoders to produce robust 3D object detection and localization in real-time. The performance of the proposed fusion model is evaluated on the popular KITTI 3D benchmark dataset

AI Patents: A Data Driven Approach

While artificial intelligence (AI) research brings challenges, the resulting systems are no accident. In fact, academics, researchers, and industry professionals have been developing AI systems since the early 1900s. AI is a field uniquely positioned at the intersection of several scientific disciplines including computer science, applied mathematics, and neuroscience. The AI design process is meticulous, deliberate, and time-consuming – involving intensive mathematical theory, data processing, and computer programming. All the while, AI’s economic value is accelerating. As such, protecting the intellectual property (IP) springing from this work is a keystone for technology firms acting in competitive markets

The Development of Regional Forest Inventories Through Novel Means

For two decades Light Detection and Ranging (LiDAR) data has been used to develop spatially-explicit forest inventories. Data derived from LiDAR depict three-dimensional forest canopy structure and are useful for predicting forest attributes such as biomass, stem density, and species. Such enhanced forest inventories (EFIs) are useful for carbon accounting, forest management, and wildlife habitat characterization by allowing practitioners to target specific areas without extensive field work. Here in New England, LiDAR data covers nearly the entire geographical extent of the region. However, until now the region’s forest attributes have not been mapped. Developing regional inventories has traditionally been problematic because most regions – including New England – are comprised of a patchwork of datasets acquired with various specifications. These variations in specifications prohibit developing a single set of predictive models for a region. The purpose of this work is to develop a new set of modeling techniques, allowing for EFIs consisting of disparate LiDAR datasets. The work presented in the first chapter improves upon existing LiDAR modeling techniques by developing a new set of metrics for quantifying LiDAR based on ecological ii principles. These fall into five categories: canopy height, canopy complexity, individual tree attributes, crowding, and abiotic. These metrics were compared to those traditionally used, and results indicated that they are a more effective means of modeling forest attributes across multiple LiDAR datasets. In the following chapters, artificial intelligence (AI) algorithms were developed to interpret LiDAR data and make forest predictions. After settling on the optimal algorithm, we incorporated satellite spectral, disturbance, and climate data. Our results indicated that this approach dramatically outperformed the traditional modeling techniques. We then applied the AI model to the region’s LiDAR, developing 10 m resolution wall-to-wall forest inventory maps of fourteen forest attributes. We assessed error using U.S. federal inventory data, and determined that our EFIs did not differ significantly in 33, 25, and 30/38 counties when predicting biomass, percent conifer, and stem density. We were ultimately able to develop the region’s most complete and detailed forest inventories. This will allow practitioners to assess forest characteristics without the cost and effort associated with extensive field-inventories