Toward Accurate Camera-based 3D Object Detection via Cascade Depth Estimation and Calibration

Recent camera-based 3D object detection is limited by the precision of transforming from image to 3D feature spaces, as well as the accuracy of object localization within the 3D space. This paper aims to address such a fundamental problem of camera-based 3D object detection: How to effectively learn depth information for accurate feature lifting and object localization. Different from previous methods which directly predict depth distributions by using a supervised estimation model, we propose a cascade framework consisting of two depth-aware learning paradigms. First, a depth estimation (DE) scheme leverages relative depth information to realize the effective feature lifting from 2D to 3D spaces. Furthermore, a depth calibration (DC) scheme introduces depth reconstruction to further adjust the 3D object localization perturbation along the depth axis. In practice, the DE is explicitly realized by using both the absolute and relative depth optimization loss to promote the precision of depth prediction, while the capability of DC is implicitly embedded into the detection Transformer through a depth denoising mechanism in the training phase. The entire model training is accomplished through an end-to-end manner. We propose a baseline detector and evaluate the effectiveness of our proposal with +2.2%/+2.7% NDS/mAP improvements on NuScenes benchmark, and gain a comparable performance with 55.9%/45.7% NDS/mAP. Furthermore, we conduct extensive experiments to demonstrate its generality based on various detectors with about +2% NDS improvements.Comment: Accepted to ICRA202

Nonmetric Trait Correlation: A Look at Environmental and Biological Influences on Third Trochanter Formation Among Pre-Contact Upper Midwest Populations

Nonmetric traits of the human skeleton are thought to correlate with genetic and/or environmental influences; however, to what extent each may affect the presence of nonmetric traits has not been clearly substantiated in the literature. Nonmetric traits as defined by Larsen are, discrete or quasi-continuous anatomical entities often expressed as gradations from absence to full expression (1997:305). More precisely, nonmetric traits are anomalies that express themselves in the skeleton and are recorded as absent or present. A third trochanter is one of many nonmetric traits present in the femur and is defined by Finnegan as, a rounded tubercle that can be found at the superior end of the gluteal crest of the femur (1978:25). The third trochanter is considered an enthesopathy as well as a nonmetric trait because it is the insertion point of the gluteus maximus muscle, the most superficial muscle in the gluteal region (Gray 1918:426). Recent studies (Hawkey and Merbs 1995, Knusel 2000) indicate that enthesopathies are closely linked to patterns of subsistence, habitual activities and geographic location. It should also be noted that enthesopathies have been directly related to pathology, trauma, biological diversity, age, hormonal, and rheumatic conditions (Hawkey and Merbs 1995, Jurmain 1999). This research will examine the correlation between sex, age, pathology, and environmental influences on the presence of third trochanters in pre-contact populations of the Upper Midwest region of the United States

SupFusion: Supervised LiDAR-Camera Fusion for 3D Object Detection

In this paper, we propose a novel training strategy called SupFusion, which provides an auxiliary feature level supervision for effective LiDAR-Camera fusion and significantly boosts detection performance. Our strategy involves a data enhancement method named Polar Sampling, which densifies sparse objects and trains an assistant model to generate high-quality features as the supervision. These features are then used to train the LiDAR-Camera fusion model, where the fusion feature is optimized to simulate the generated high-quality features. Furthermore, we propose a simple yet effective deep fusion module, which contiguously gains superior performance compared with previous fusion methods with SupFusion strategy. In such a manner, our proposal shares the following advantages. Firstly, SupFusion introduces auxiliary feature-level supervision which could boost LiDAR-Camera detection performance without introducing extra inference costs. Secondly, the proposed deep fusion could continuously improve the detector's abilities. Our proposed SupFusion and deep fusion module is plug-and-play, we make extensive experiments to demonstrate its effectiveness. Specifically, we gain around 2% 3D mAP improvements on KITTI benchmark based on multiple LiDAR-Camera 3D detectors.Comment: Accepted to ICCV202

Quantitative Bioluminescence Tomography-guided System for Conformal Irradiation In Vivo

Although cone-beam CT (CBCT) has been used to guide irradiation for pre-clinical radiotherapy(RT) research, it is limited to localize soft tissue target especially in a low imaging contrast environment. Knowledge of target shape is a fundamental need for RT. Without such information to guide radiation, normal tissue can be irradiated unnecessarily, leading to experimental uncertainties. Recognition of this need led us to develop quantitative bioluminescence tomography (QBLT), which provides strong imaging contrast to localize optical targets. We demonstrated its capability of guiding conformal RT using an orthotopic bioluminescent glioblastoma (GBM) model. With multi-projection and multi-spectral bioluminescence imaging and a novel spectral derivative method, our QBLT system is able to reconstruct GBM with localization accuracy <1mm. An optimal threshold was determined to delineate QBLT reconstructed gross target volume (GTV_{QBLT}), which provides the best overlap between the GTV_{QBLT} and CBCT contrast labeled GBM (GTV), used as the ground truth for the GBM volume. To account for the uncertainty of QBLT in target localization and volume delineation, we also innovated a margin design; a 0.5mm margin was determined and added to GTV_{QBLT} to form a planning target volume (PTV_{QBLT}), which largely improved tumor coverage from 75% (0mm margin) to 98% and the corresponding variation (n=10) of the tumor coverage was significantly reduced. Moreover, with prescribed dose 5Gy covering 95% of PTV_{QBLT}, QBLT-guided 7-field conformal RT can irradiate 99.4 \pm 1.0% of GTV vs. 65.5 \pm 18.5% with conventional single field irradiation (n=10). Our QBLT-guided system provides a unique opportunity for researchers to guide irradiation for soft tissue targets and increase rigorous and reproducibility of scientific discovery

Arc-to-line frame registration method for ultrasound and photoacoustic image-guided intraoperative robot-assisted laparoscopic prostatectomy

Purpose: To achieve effective robot-assisted laparoscopic prostatectomy, the integration of transrectal ultrasound (TRUS) imaging system which is the most widely used imaging modelity in prostate imaging is essential. However, manual manipulation of the ultrasound transducer during the procedure will significantly interfere with the surgery. Therefore, we propose an image co-registration algorithm based on a photoacoustic marker method, where the ultrasound / photoacoustic (US/PA) images can be registered to the endoscopic camera images to ultimately enable the TRUS transducer to automatically track the surgical instrument Methods: An optimization-based algorithm is proposed to co-register the images from the two different imaging modalities. The principles of light propagation and an uncertainty in PM detection were assumed in this algorithm to improve the stability and accuracy of the algorithm. The algorithm is validated using the previously developed US/PA image-guided system with a da Vinci surgical robot. Results: The target-registration-error (TRE) is measured to evaluate the proposed algorithm. In both simulation and experimental demonstration, the proposed algorithm achieved a sub-centimeter accuracy which is acceptable in practical clinics. The result is also comparable with our previous approach, and the proposed method can be implemented with a normal white light stereo camera and doesn't require highly accurate localization of the PM. Conclusion: The proposed frame registration algorithm enabled a simple yet efficient integration of commercial US/PA imaging system into laparoscopic surgical setting by leveraging the characteristic properties of acoustic wave propagation and laser excitation, contributing to automated US/PA image-guided surgical intervention applications.Comment: 12 pages, 9 figure

Deep muscularis propria tumor invasion without lymph node metastasis as a unique subclassification of stage IB gastric cancer: a retrospective study

BACKGROUND: The prognosis difference based on the depth of tumor muscularis propria invasion in gastric cancer (GC) was still debated, and therapy strategy for stage IB GC patient required further investigation. METHODS: A total of 380 patients with pT2 GC after radical surgery were retrospectively analyzed, including 185 in superficial muscularis propria (sMP) group and 195 in deep muscularis propria (dMP) group. RESULTS: The overall survival (OS) was significantly better for patients in sMP group than for patients in dMP group (P = 0.007). In multivariate analysis, depth of tumor invasion, pN stage, age, primary location, positive expression of p53, elevated maximal LDH, elevated initial CA19-9 and AFP level were independent prognostic factors for OS. The sMP group had a significantly better OS than dMP group (P = 0.014) in pN0 stage. After further stratification, the survival outcomes were not significantly different between deep muscularis propria tumor invasion without lymph node metastasis (dMPN0) group (stage IB) and superficial muscularis propria tumor invasion with stage 1-2 lymph node metastasis (sMPN1-2) group (stage II) (P = 0.100). Patients with adjuvant chemotherapy had a statistically better survival than those without in dMPN0 group (P = 0.045) and dMPN0 patients with adjuvant chemotherapy had better OS than sMPN1-2 patients (P = 0.015). In addition, greater postoperative survival could be observed in sMPN0 patients than dMPN0 patients in p53-positive group (P = 0.002), and similar OS could be seen between dMPN0 patients with p53-positive and T2N1-2 patients (P = 0.872). CONCLUSION: As a unique subclassification of stage IB GC, appropriate adjuvant chemotherapy should be considered for patients with dMPN0 stage. In addition, positive expression of p53, elevated LDH could be potential factors in identifying the different prognoses for stage IB GC patients