Multi-Modal 3D Object Detection in Autonomous Driving: a Survey

In the past few years, we have witnessed rapid development of autonomous driving. However, achieving full autonomy remains a daunting task due to the complex and dynamic driving environment. As a result, self-driving cars are equipped with a suite of sensors to conduct robust and accurate environment perception. As the number and type of sensors keep increasing, combining them for better perception is becoming a natural trend. So far, there has been no indepth review that focuses on multi-sensor fusion based perception. To bridge this gap and motivate future research, this survey devotes to review recent fusion-based 3D detection deep learning models that leverage multiple sensor data sources, especially cameras and LiDARs. In this survey, we first introduce the background of popular sensors for autonomous cars, including their common data representations as well as object detection networks developed for each type of sensor data. Next, we discuss some popular datasets for multi-modal 3D object detection, with a special focus on the sensor data included in each dataset. Then we present in-depth reviews of recent multi-modal 3D detection networks by considering the following three aspects of the fusion: fusion location, fusion data representation, and fusion granularity. After a detailed review, we discuss open challenges and point out possible solutions. We hope that our detailed review can help researchers to embark investigations in the area of multi-modal 3D object detection

The large area detector onboard the eXTP mission

The Large Area Detector (LAD) is the high-throughput, spectral-timing instrument onboard the eXTP mission, a flagship mission of the Chinese Academy of Sciences and the China National Space Administration, with a large European participation coordinated by Italy and Spain. The eXTP mission is currently performing its phase B study, with a target launch at the end-2027. The eXTP scientific payload includes four instruments (SFA, PFA, LAD and WFM) offering unprecedented simultaneous wide-band X-ray timing and polarimetry sensitivity. The LAD instrument is based on the design originally proposed for the LOFT mission. It envisages a deployed 3.2 m2 effective area in the 2-30 keV energy range, achieved through the technology of the large-area Silicon Drift Detectors - offering a spectral resolution of up to 200 eV FWHM at 6 keV - and of capillary plate collimators - limiting the field of view to about 1 degree. In this paper we will provide an overview of the LAD instrument design, its current status of development and anticipated performance

The large area detector onboard the eXTP mission

The eXTP (enhanced X-ray Timing and Polarimetry) mission is a major project of the Chinese Academy of Sciences (CAS) and China National Space Administration (CNSA) currently performing an extended phase A study and proposed for a launch by 2025 in a low-earth orbit. The eXTP scientific payload envisages a suite of instruments (Spectroscopy Focusing Array, Polarimetry Focusing Array, Large Area Detector and Wide Field Monitor) offering unprecedented simultaneous wide-band X-ray spectral, timing and polarimetry sensitivity. A large European consortium is contributing to the eXTP study and it is expected to provide key hardware elements, including a Large Area Detector (LAD). The LAD instrument for eXTP is based on the design originally proposed for the LOFT mission within the ESA context. The eXTP/LAD envisages a deployed 3.4 m2 effective area in the 2-30 keV energy range, achieved through the technology of the large-area Silicon Drift Detectors - offering a spectral resolution of up to 200 eV FWHM at 6 keV - and of capillary plate collimators - limiting the field of view to about 1 degree. In this paper we provide an overview of the LAD instrument design, including new elements with respect to the earlier LOFT configuration

The large area detector onboard the eXTP mission

The Large Area Detector (LAD) is the high-throughput, spectral-timing instrument onboard the eXTP mission, a flagship mission of the Chinese Academy of Sciences and the China National Space Administration, with a large European participation coordinated by Italy and Spain. The eXTP mission is currently performing its phase B study, with a target launch at the end-2027. The eXTP scientific payload includes four instruments (SFA, PFA, LAD and WFM) offering unprecedented simultaneous wide-band X-ray timing and polarimetry sensitivity. The LAD instrument is based on the design originally proposed for the LOFT mission. It envisages a deployed 3.2 m2 effective area in the 2-30 keV energy range, achieved through the technology of the large-area Silicon Drift Detectors - offering a spectral resolution of up to 200 eV FWHM at 6 keV - and of capillary plate collimators - limiting the field of view to about 1 degree. In this paper we will provide an overview of the LAD instrument design, its current status of development and anticipated performance