459 research outputs found

    Lidar-based Obstacle Detection and Recognition for Autonomous Agricultural Vehicles

    Get PDF
    Today, agricultural vehicles are available that can drive autonomously and follow exact route plans more precisely than human operators. Combined with advancements in precision agriculture, autonomous agricultural robots can reduce manual labor, improve workflow, and optimize yield. However, as of today, human operators are still required for monitoring the environment and acting upon potential obstacles in front of the vehicle. To eliminate this need, safety must be ensured by accurate and reliable obstacle detection and avoidance systems.In this thesis, lidar-based obstacle detection and recognition in agricultural environments has been investigated. A rotating multi-beam lidar generating 3D point clouds was used for point-wise classification of agricultural scenes, while multi-modal fusion with cameras and radar was used to increase performance and robustness. Two research perception platforms were presented and used for data acquisition. The proposed methods were all evaluated on recorded datasets that represented a wide range of realistic agricultural environments and included both static and dynamic obstacles.For 3D point cloud classification, two methods were proposed for handling density variations during feature extraction. One method outperformed a frequently used generic 3D feature descriptor, whereas the other method showed promising preliminary results using deep learning on 2D range images. For multi-modal fusion, four methods were proposed for combining lidar with color camera, thermal camera, and radar. Gradual improvements in classification accuracy were seen, as spatial, temporal, and multi-modal relationships were introduced in the models. Finally, occupancy grid mapping was used to fuse and map detections globally, and runtime obstacle detection was applied on mapped detections along the vehicle path, thus simulating an actual traversal.The proposed methods serve as a first step towards full autonomy for agricultural vehicles. The study has thus shown that recent advancements in autonomous driving can be transferred to the agricultural domain, when accurate distinctions are made between obstacles and processable vegetation. Future research in the domain has further been facilitated with the release of the multi-modal obstacle dataset, FieldSAFE

    Spatio-Temporal Terrain Classification for Mapping and Robot Localization

    Get PDF
    Detection and classification of the surrounding terrain is a fundamental ability of a mobile robot in outdoor navigation to enable safe and efficient path planning. Our robot is equipped with a 3D LiDAR and a color camera, since these sensors complement each other very well. The terrain in front of the robot is divided into a grid, and each grid cell is classified individually using the sensor measurements. A new method for 3D LiDAR-based terrain classification with easy-to-compute and yet discriminative 3D features based on intensity and roughness histograms is presented. To exploit the fact that terrain appears in contiguous areas, spatial dependencies between the individual cells of the terrain grid are taken into account by modeling the grid as a Conditional random field. As the robot moves, we constantly update a terrain map with the current classification result. In this way, we are not only able to exploit temporal dependencies, but we are building whole terrain maps of the environment. Finally, we show how to use these maps for a semantic localization of the robot

    Méthode de fusion multi-capteurs pour des opérations de suivi de lignes de culture et de traversabilité

    Get PDF
    Conférence AXEMA-EURAGENG 2017, Paris, FRA, 25-/02/2017 - 25/02/2017International audiencePrecision agriculture vehicles need autonomous navigation in cultures to carry out their tasks, such as planting, maintenance and harvesting in cultures such as vegetable, vineyard, or horticulture. The detection of natural objects like trunks, grass, leaf, or obstacles in front of vehicle in crop row is crucial for safe navigation. Sensors such as LiDAR devices or Time Of Flight cameras (TOF), allow to obtain geometric data in natural environment, using information of an Inertial Measurement Unit (IMU), for measurement accuracy. Fusion of geometric information with a color camera data improves the natural object identification, using some color classification technique, such as Support Vector Machine (SVM), considering two object classes, either solid objects such as crop or tree branch, and other elements like grass, leaf and soil. Agricultural vehicles can use these geometric and colorimetric data in real time, to follow crop rows and detect obstacles while executing various precision agriculture operations. In this application, perception sensors embedded on a light mobile robot were used to detect and identify natural objects in agricultural crops, working in various fields, with or without soil perturbation, with different speeds and several vegetation levels to achieve crop row tracking tasks, from a desired lateral deviation between robot and crop line, or traversability operations which consisted to take a decision in vehicle navigation, according to the size and nature of the detected objects, in front of vehicle. The vehicle could cross or avoid the object, or it must stop, for big solid obstacles

    Recent Advances in Image Restoration with Applications to Real World Problems

    Get PDF
    In the past few decades, imaging hardware has improved tremendously in terms of resolution, making widespread usage of images in many diverse applications on Earth and planetary missions. However, practical issues associated with image acquisition are still affecting image quality. Some of these issues such as blurring, measurement noise, mosaicing artifacts, low spatial or spectral resolution, etc. can seriously affect the accuracy of the aforementioned applications. This book intends to provide the reader with a glimpse of the latest developments and recent advances in image restoration, which includes image super-resolution, image fusion to enhance spatial, spectral resolution, and temporal resolutions, and the generation of synthetic images using deep learning techniques. Some practical applications are also included

    TractorEYE: Vision-based Real-time Detection for Autonomous Vehicles in Agriculture

    Get PDF
    Agricultural vehicles such as tractors and harvesters have for decades been able to navigate automatically and more efficiently using commercially available products such as auto-steering and tractor-guidance systems. However, a human operator is still required inside the vehicle to ensure the safety of vehicle and especially surroundings such as humans and animals. To get fully autonomous vehicles certified for farming, computer vision algorithms and sensor technologies must detect obstacles with equivalent or better than human-level performance. Furthermore, detections must run in real-time to allow vehicles to actuate and avoid collision.This thesis proposes a detection system (TractorEYE), a dataset (FieldSAFE), and procedures to fuse information from multiple sensor technologies to improve detection of obstacles and to generate a map. TractorEYE is a multi-sensor detection system for autonomous vehicles in agriculture. The multi-sensor system consists of three hardware synchronized and registered sensors (stereo camera, thermal camera and multi-beam lidar) mounted on/in a ruggedized and water-resistant casing. Algorithms have been developed to run a total of six detection algorithms (four for rgb camera, one for thermal camera and one for a Multi-beam lidar) and fuse detection information in a common format using either 3D positions or Inverse Sensor Models. A GPU powered computational platform is able to run detection algorithms online. For the rgb camera, a deep learning algorithm is proposed DeepAnomaly to perform real-time anomaly detection of distant, heavy occluded and unknown obstacles in agriculture. DeepAnomaly is -- compared to a state-of-the-art object detector Faster R-CNN -- for an agricultural use-case able to detect humans better and at longer ranges (45-90m) using a smaller memory footprint and 7.3-times faster processing. Low memory footprint and fast processing makes DeepAnomaly suitable for real-time applications running on an embedded GPU. FieldSAFE is a multi-modal dataset for detection of static and moving obstacles in agriculture. The dataset includes synchronized recordings from a rgb camera, stereo camera, thermal camera, 360-degree camera, lidar and radar. Precise localization and pose is provided using IMU and GPS. Ground truth of static and moving obstacles (humans, mannequin dolls, barrels, buildings, vehicles, and vegetation) are available as an annotated orthophoto and GPS coordinates for moving obstacles. Detection information from multiple detection algorithms and sensors are fused into a map using Inverse Sensor Models and occupancy grid maps. This thesis presented many scientific contribution and state-of-the-art within perception for autonomous tractors; this includes a dataset, sensor platform, detection algorithms and procedures to perform multi-sensor fusion. Furthermore, important engineering contributions to autonomous farming vehicles are presented such as easily applicable, open-source software packages and algorithms that have been demonstrated in an end-to-end real-time detection system. The contributions of this thesis have demonstrated, addressed and solved critical issues to utilize camera-based perception systems that are essential to make autonomous vehicles in agriculture a reality

    Automated 3D object modeling from aerial video imagery

    Get PDF
    Research in physically accurate 3D modeling of a scene is gaining momentum because of its far reaching applications in civilian and defense sectors. The modeled 3D scene must conform both geometrically and spectrally to the real world for all the applications. Geometric modeling of a scene can be achieved in many ways of which the two most popular methods are - a) using multiple 2D passive images of the scene also called as stereo vision and b) using 3D point clouds like Lidar (Light detection and ranging) data. In this research work, we derive the 3D models of objects in a scene using passive aerial video imagery. At present, this geometric modeling requires a lot of manual intervention due to a variety of factors like sensor noise, low contrast conditions during image capture, etc. Hence long time periods, in the order of weeks and months, are required to model even a small scene. This thesis focuses on automating the process of geometric modeling of objects in a scene from passive aerial video imagery. The aerial video frames are stitched into stereo mosaics. These stereo mosaics not only provide the elevation information of a scene but also act as good 3D visualization tools. The 3D information obtained from the stereo mosaics is used to identify the various 3D objects, especially man-made buildings using probabilistic inference provided by Bayesian Networks. The initial 3D building models are further optimized by projecting them on to the individual video frames. The limitations of the state-of-art technology in attaining these goals are presented along with the techniques to overcome them. The improvement that can be achieved in the accuracy of the 3D models when Lidar data is fused with aerial video during the object identification process is also examined
    • …
    corecore