Attentional PointNet for 3D-Object Detection in Point Clouds

International audienc

Attention Network for 3D Object Detection in Point Clouds

International audienceAccurate detection of objects in 3D point clouds is a central problem for autonomous navigation. Most existing methods use techniques of handcrafted features representation or multi-modal approaches prone to sensor failure. Approaches like PointNet that directly operate on sparse point data have shown good accuracy in the classification of single 3D objects. However, LiDAR sensors on Autonomous vehicles generate a large scale pointcloud. Real-time object detection in such a cluttered environment still remains a challenge. In this thesis, we propose Attentional PointNet, a novel end-toend trainable deep architecture for object detection in point clouds. We extend the theory of visual attention mechanism to 3D point clouds and introduce a new recurrent 3D Spatial Transformer Network module. Rather than processing whole point cloud, the network learns "where to look" (find regions of interest), thus significantly reducing the number of points and hence, inference time. Evaluation on KITTI car detection benchmark shows that our Attentional PointNet is notably faster and achieves comparable results with state-of-the-art LiDAR-based 3D detection methods.La détection précise d’objets dans un nuage de points 3D est un problème central pour la navigation autonome. La plupart des méthodes existantes utilisent des caractéristiques sélectionnées à la main ou des approches multimodèlessujettes à une défaillance du capteur. Des approches, telles que PointNet fonctionnant directement sur des données ponctuelles éparses, classifient précisément un nuage de points associé à un unique objet. Cependant, les capteurs Lidars sur les véhicules autonomes génèrent un nuage de points contenant de nombreux objets. Leurs détections en temps réel dans un environnement aussi encombré restent un défi. Dans cette thèse, nous proposons une méthode appelée Attentional PointNet, une architecture profonde complète, formable de bout en bout, destinée à la détection d’objets dans le nuage de points. Nous étendons la théorie du mécanisme d’attention visuelle au nuage de points 3D et introduisons un nouveau module récurrent de réseau de transformateur spatial 3D. Plutôt que de traiter le nuage de points dans sont ensemble, il apprend à reconnaître des régions potentiellement intéressantes. Ensuite, localiser des objets dans ces régions réduit considérablement le nombre de points à traiter et réduit le temps de calcul. L’évaluation avec les données du jeu de données KITTI montre que notre méthode est plus rapide et permet d’obtenir des résultats comparables avec les méthodes classiques de détection 3D utilisant des nuages de points générés par des Lidars

A Study of Attention-Free and Attentional Methods for LiDAR and 4D Radar Object Detection in Self-Driving Applications

In this thesis, we re-examine the problem of 3D object detection in the context of self driving cars with the first publicly released View of Delft (VoD) dataset [1] containing 4D radar sensor data. 4D radar is a novel sensor that provides velocity and Radar Cross Section (RCS) information in addition to position for its point cloud. State of the art architectures such as 3DETR [2] and IASSD [3] were used as a baseline. Several attention-free methods, like point cloud concatenation, feature propagation and feature fusion with MLP, as well as attentional methods utilizing cross attention, were tested to determine how we can best combine LiDAR and radar to develop a multimodal detection architecture that outperforms the baseline architectures trained only on either modality alone. Our findings indicate that while attention-free methods did not consistently surpass the baseline performance across all classes, they did lead to notable performance gains for specific classes. Furthermore, we found that attentional methods faced challenges due to the sparsity of radar point clouds and duplicated features, which limited the efficacy of the crossattention mechanism. These findings highlight potential avenues for future research to refine and improve upon attentional methods in the context of 3D object detection

3D objects and scenes classification, recognition, segmentation, and reconstruction using 3D point cloud data: A review

Three-dimensional (3D) point cloud analysis has become one of the attractive subjects in realistic imaging and machine visions due to its simplicity, flexibility and powerful capacity of visualization. Actually, the representation of scenes and buildings using 3D shapes and formats leveraged many applications among which automatic driving, scenes and objects reconstruction, etc. Nevertheless, working with this emerging type of data has been a challenging task for objects representation, scenes recognition, segmentation, and reconstruction. In this regard, a significant effort has recently been devoted to developing novel strategies, using different techniques such as deep learning models. To that end, we present in this paper a comprehensive review of existing tasks on 3D point cloud: a well-defined taxonomy of existing techniques is performed based on the nature of the adopted algorithms, application scenarios, and main objectives. Various tasks performed on 3D point could data are investigated, including objects and scenes detection, recognition, segmentation and reconstruction. In addition, we introduce a list of used datasets, we discuss respective evaluation metrics and we compare the performance of existing solutions to better inform the state-of-the-art and identify their limitations and strengths. Lastly, we elaborate on current challenges facing the subject of technology and future trends attracting considerable interest, which could be a starting point for upcoming research studie

PointGrow: Autoregressively Learned Point Cloud Generation with Self-Attention

Generating 3D point clouds is challenging yet highly desired. This work presents a novel autoregressive model, PointGrow, which can generate diverse and realistic point cloud samples from scratch or conditioned on semantic contexts. This model operates recurrently, with each point sampled according to a conditional distribution given its previously-generated points, allowing inter-point correlations to be well-exploited and 3D shape generative processes to be better interpreted. Since point cloud object shapes are typically encoded by long-range dependencies, we augment our model with dedicated self-attention modules to capture such relations. Extensive evaluations show that PointGrow achieves satisfying performance on both unconditional and conditional point cloud generation tasks, with respect to realism and diversity. Several important applications, such as unsupervised feature learning and shape arithmetic operations, are also demonstrated