CoBEV: Elevating Roadside 3D Object Detection with Depth and Height Complementarity

Roadside camera-driven 3D object detection is a crucial task in intelligent transportation systems, which extends the perception range beyond the limitations of vision-centric vehicles and enhances road safety. While previous studies have limitations in using only depth or height information, we find both depth and height matter and they are in fact complementary. The depth feature encompasses precise geometric cues, whereas the height feature is primarily focused on distinguishing between various categories of height intervals, essentially providing semantic context. This insight motivates the development of Complementary-BEV (CoBEV), a novel end-to-end monocular 3D object detection framework that integrates depth and height to construct robust BEV representations. In essence, CoBEV estimates each pixel's depth and height distribution and lifts the camera features into 3D space for lateral fusion using the newly proposed two-stage complementary feature selection (CFS) module. A BEV feature distillation framework is also seamlessly integrated to further enhance the detection accuracy from the prior knowledge of the fusion-modal CoBEV teacher. We conduct extensive experiments on the public 3D detection benchmarks of roadside camera-based DAIR-V2X-I and Rope3D, as well as the private Supremind-Road dataset, demonstrating that CoBEV not only achieves the accuracy of the new state-of-the-art, but also significantly advances the robustness of previous methods in challenging long-distance scenarios and noisy camera disturbance, and enhances generalization by a large margin in heterologous settings with drastic changes in scene and camera parameters. For the first time, the vehicle AP score of a camera model reaches 80% on DAIR-V2X-I in terms of easy mode. The source code will be made publicly available at https://github.com/MasterHow/CoBEV.Comment: The source code will be made publicly available at https://github.com/MasterHow/CoBE

Technology Roadmap for Flexible Sensors

Humans rely increasingly on sensors to address grand challenges and to improve quality of life in the era of digitalization and big data. For ubiquitous sensing, flexible sensors are developed to overcome the limitations of conventional rigid counterparts. Despite rapid advancement in bench-side research over the last decade, the market adoption of flexible sensors remains limited. To ease and to expedite their deployment, here, we identify bottlenecks hindering the maturation of flexible sensors and propose promising solutions. We first analyze challenges in achieving satisfactory sensing performance for real-world applications and then summarize issues in compatible sensor-biology interfaces, followed by brief discussions on powering and connecting sensor networks. Issues en route to commercialization and for sustainable growth of the sector are also analyzed, highlighting environmental concerns and emphasizing nontechnical issues such as business, regulatory, and ethical considerations. Additionally, we look at future intelligent flexible sensors. In proposing a comprehensive roadmap, we hope to steer research efforts towards common goals and to guide coordinated development strategies from disparate communities. Through such collaborative efforts, scientific breakthroughs can be made sooner and capitalized for the betterment of humanity

Technology roadmap for flexible sensors

Humans rely increasingly on sensors to address grand challenges and to improve quality of life in the era of digitalization and big data. For ubiquitous sensing, flexible sensors are developed to overcome the limitations of conventional rigid counterparts. Despite rapid advancement in bench-side research over the last decade, the market adoption of flexible sensors remains limited. To ease and to expedite their deployment, here, we identify bottlenecks hindering the maturation of flexible sensors and propose promising solutions. We first analyze challenges in achieving satisfactory sensing performance for real-world applications and then summarize issues in compatible sensor-biology interfaces, followed by brief discussions on powering and connecting sensor networks. Issues en route to commercialization and for sustainable growth of the sector are also analyzed, highlighting environmental concerns and emphasizing nontechnical issues such as business, regulatory, and ethical considerations. Additionally, we look at future intelligent flexible sensors. In proposing a comprehensive roadmap, we hope to steer research efforts towards common goals and to guide coordinated development strategies from disparate communities. Through such collaborative efforts, scientific breakthroughs can be made sooner and capitalized for the betterment of humanity.Agency for Science, Technology and Research (A*STAR)National Research Foundation (NRF)Submitted/Accepted versionY.L., Z.L., M.Z., and X.C. acknowledge the National Research Foundation, Singapore (NRF) under NRF’s Medium Sized Centre: Singapore Hybrid-Integrated Next-Generation μElectronics (SHINE) Centre funding programme, and AME programming funding scheme of Cyber Physiochemical Interface (CPI) project (no. A18A1b0045). Y.L. acknowledges National Natural Science Foundation of China (62201243). C.J. acknowledges funding support from the National Key R&D Program of China (no. 2019YFA0706100), the National Natural Science Foundation of China (82151305), Lingang Laboratory (LG-QS-202202-09). T.Q.T. and N.E.L. acknowledge support by the Basic Science Research Program (no. 2020R1A2C3013480) through the National Research Foundation of Korea (NRF) funded by the Ministry of Science and ICT. A.F. acknowledges the AFOSR (grant FA9550-22-1-0423). Y.L. and Y.Z. would like to acknowledge the NSF (award no. 2134664) and NIH (award no. R01HD108473) for financial support. X.F. acknowledges the support from the National Natural Science Foundation of China (grant no. U20A6001). L.Y. would like to thank the A*STAR Central Research Fund (CRF) and the AME Programmatic A18A1b0045 (Cyber Physiochemical Interfaces) for funding support. C.F.G. acknowledges the National Natural Science Foundation of China (no. T2225017). T.Q.T. acknowledges the Brain Pool Program (No. 2020H1D3A2A02111068) through the National Research Foundation (NRF) funded by the Ministry of Science. Z.L. acknowledges the support from RIE2020 AME Programmatic Grant funded by A*STAR-SERC, Singapore (Grant No. A18A1b0045). X.G. acknowledges funding support through the Shanghai Science and Technology Commission (grant no. 19JC1412400), the National Science Fund for Excellent Young Scholars (grant no. 61922057). C.D. acknowledges National Science Foundation CAREER: Conformable Piezoelectrics for Soft Tissue Imaging (grant no. 2044688) and MIT Media Lab Consortium funding. D.K. and O.G.S. acknowledge Leibniz Association and the German Research Foundation DFG (Gottfried Wilhelm Leibniz Program SCHM 1298/22-1, KA5051/1-1 and KA 5051/3-1), as well as the Leibniz association (Leibniz Transfer Program T62/2019). C.W. acknowledges the National Key Research and Development Program of China (grant no. 2021YFA1202600), National Natural Science Foundation of China (grant no. 62174082). A.V.-Y.T., E.Z., Y.Z., X.Z., and J.P. acknowledge the National Research Foundation, Singapore (NRF) under NRF’s Medium Sized Centre: Singapore Hybrid-Integrated Next-Generation μElectronics (SHINE) Centre funding programme, and AME programming funding scheme of Cyber Physiochemical Interface (CPI) project (no. A18A1b0045). R.Z. acknowledges National Natural Science Foundation of China (grant no. 51735007) and Beijing Natural Science Foundation (grant no. 3191001). N.M. acknowledges the support by JST PRESTO Grant Number JPMJPR20B7 and JST Adaptable and Seamless Technology transfer Program through Target-driven R&D (ASTEP) grant number JPMJTM22BK. C.P. acknowledges the Korean government (Ministry of Science and ICT, MSIT) (2022R1A4A3032923). M.W. acknowledges the National Key R&D Program of China under Grant (2021YFB3601200). X.Z. acknowledges National Natural Science Foundation of China (no. 62074029). S.X. acknowledges the 3M nontenured faculty award. T.-W.L. and D.-G.S. acknowledge the Pioneer Research Center Program through the National Research Foundation of Korea funded by the Ministry of Science, ICT & Future Planning (grant no. NRF-2022M3C1A3081211). C.T.L. would like to acknowledge support from the Institute for Health Innovation and Technology (iHealthtech), the MechanoBioEngineering Laboratory at the Department of Biomedical Engineering and the Institute for Functional Intelligent Materials (I-FIM) at the National University of Singapore (NUS). C.T.L. also acknowledges support from the National Research Foundation and A*STAR, under its RIE2020 Industry Alignment Fund − Industry Collaboration Projects (IAF-ICP) (grant no. I2001E0059) − SIA-NUS Digital Aviation Corp Lab and the NUS ARTIC Research (grant no. HFM-RP1). X.Y. acknowledges funding support by City University of Hong Kong (grant no. 9667221). T.X. and X.Z. acknowledge National Natural Science Foundation of China (22234006). B.C.K.T. acknowledges Cyber-Physiochemical Interfaces CPI, A*STAR A18A1b0045. H.G. acknowledges a research start-up grant (002479-00001) from Nanyang Technological University and the Agency for Science, Technology and Research (A*STAR) in Singapore. W.G. acknowledges National Science Foundation grant 2145802. D.J.L. acknowledges support from the US National Science Foundation grant number CBET-2223566. G.Y. acknowledges support from The Welch Foundation award F-1861, and Camille Dreyfus Teacher-Scholar Award. M.D.D. acknowledges funding support from NSF (grant no. EEC1160483). J.-H.A acknowledges the National Research Foundation of Korea (NRF-2015R1A3A2066337). J.C. acknowledges the Henry Samueli School of Engineering & Applied Science and the Department of Bioengineering at the University of California, Los Angeles for startup support and a Brain & Behavior Research Foundation Young Investigator Grant. K.T. acknowledges JST AIP Accelerated Program (no. JPMJCR21U1) and JSPS KAKENHI (grant no. JP22H00594). P.S.W. acknowledges the National Science Foundation (CMMI1636136) for support. A.M.A., M.C.H., and P.S.W. thank the National Institute on Drug Abuse (DA045550) for support. S.M. and X.C. appreciated the support from the Smart Grippers for Soft Robotics (SGSR) Programme under the National Research Foundation, Prime Minister’s Office, Singapore under its Campus of Research Excellence and Technological Enterprise (CREATE) programme