Current efficient LiDAR-based detection frameworks are lacking in exploitingobject relations, which naturally present in both spatial and temporal manners.To this end, we introduce a simple, efficient, and effective two-stagedetector, termed as Ret3D. At the core of Ret3D is the utilization of novelintra-frame and inter-frame relation modules to capture the spatial andtemporal relations accordingly. More Specifically, intra-frame relation module(IntraRM) encapsulates the intra-frame objects into a sparse graph and thusallows us to refine the object features through efficient message passing. Onthe other hand, inter-frame relation module (InterRM) densely connects eachobject in its corresponding tracked sequences dynamically, and leverages suchtemporal information to further enhance its representations efficiently througha lightweight transformer network. We instantiate our novel designs of IntraRMand InterRM with general center-based or anchor-based detectors and evaluatethem on Waymo Open Dataset (WOD). With negligible extra overhead, Ret3Dachieves the state-of-the-art performance, being 5.5% and 3.2% higher than therecent competitor in terms of the LEVEL 1 and LEVEL 2 mAPH metrics on vehicledetection, respectively.<br
