Practical Collaborative Perception: A Framework for Asynchronous and Multi-Agent 3D Object Detection

In this paper, we improve the single-vehicle 3D object detection models using LiDAR by extending their capacity to process point cloud sequences instead of individual point clouds. In this step, we extend our previous work on rectification of the shadow effect in the concatenation of point clouds to boost the detection accuracy of multi-frame detection models. Our extension includes incorporating HD Map and distilling an Oracle model. Next, we further increase the performance of single-vehicle perception using multi-agent collaboration via Vehicle-to-everything (V2X) communication. We devise a simple yet effective collaboration method that achieves better bandwidth-performance tradeoffs than prior arts while minimizing changes made to single-vehicle detection models and assumptions on inter-agent synchronization. Experiments on the V2X-Sim dataset show that our collaboration method achieves 98% performance of the early collaboration while consuming the equivalent amount of bandwidth usage of late collaboration which is 0.03% of early collaboration. The code will be released at https://github.com/quan-dao/practical-collab-perception.Comment: Work in progres

Cloud base height estimates from sky imagery and a network of pyranometers

Cloud base height (CBH) is an important parameter for physics-based high resolution solar radiation modeling. In sky imager-based forecasts, a ceilometer or stereographic setup is needed to derive the CBH; otherwise erroneous CBHs lead to incorrect physical cloud velocity and incorrect projection of cloud shadows, causing solar power forecast errors due to incorrect shadow positions and timing of shadowing events. In this paper, two methods to estimate cloud base height from a single sky imager and distributed ground solar irradiance measurements are proposed. The first method (Time Series Correlation, denoted as “TSC”) is based upon the correlation between ground-observed global horizontal irradiance (GHI) time series and a modeled GHI time series generated from a sequence of sky images geo-rectified to a candidate set of CBH. The estimated CBH is taken as the candidate that produces the highest correlation coefficient. The second method (Geometric Cloud Shadow Edge, denoted as “GCSE”) integrates a numerical ramp detection method for ground-observed GHI time series with solar and cloud geometry applied to cloud edges in a sky image. CBH are benchmarked against a collocated ceilometer and stereographically estimated CBH from two sky imagers for 15 min median-filtered CBHs. Over 30 days covering all seasons, the TSC method performs similarly to the GCSE method with nRMSD of 18.9% versus 20.8%. A key limitation of both proposed methods is the requirement of sufficient variation in GHI to enable reliable correlation and ramp detection. The advantage of the two proposed methods is that they can be applied when measurements from only a single sky imager and pyranometers are available

3D MODELING of A COMPLEX BUILDING: From MULTI-VIEW IMAGE FUSION to GOOGLE EARTH PUBLICATION

This paper presents a pipeline that aims at illustrating the procedure to realize a 3D model of a complex building integrating the UAV and terrestrial images and modifying the 3D model in order to publish to Google Earth in an interactive modality so as to provide better available models for visualization and use. The main steps of the procedure are the optimization of the UAV flight, the integration of the different UAV and ground floor images and the optimization of the model to be published to GE. The case study has been identified in a building, The Eremo di Santa Rosalia Convent in Sicily which hash more staggered elevations and located in the hills of the hinterland and of which, the online platform only indicate the position on Google Maps (GM) and Google Earth (GE) with a photo from above and a non-urban road whose GM path is not corresponding with the GE photo. The process highlights the integration of the models and showcases a workflow for the publication of the combined 3D model to the GE platform