Estimating 3D Uncertainty Field: Quantifying Uncertainty for Neural Radiance Fields

Current methods based on Neural Radiance Fields (NeRF) significantly lack the capacity to quantify uncertainty in their predictions, particularly on the unseen space including the occluded and outside scene content. This limitation hinders their extensive applications in robotics, where the reliability of model predictions has to be considered for tasks such as robotic exploration and planning in unknown environments. To address this, we propose a novel approach to estimate a 3D Uncertainty Field based on the learned incomplete scene geometry, which explicitly identifies these unseen regions. By considering the accumulated transmittance along each camera ray, our Uncertainty Field infers 2D pixel-wise uncertainty, exhibiting high values for rays directly casting towards occluded or outside the scene content. To quantify the uncertainty on the learned surface, we model a stochastic radiance field. Our experiments demonstrate that our approach is the only one that can explicitly reason about high uncertainty both on 3D unseen regions and its involved 2D rendered pixels, compared with recent methods. Furthermore, we illustrate that our designed uncertainty field is ideally suited for real-world robotics tasks, such as next-best-view selection

Stochastic neural radiance fields: quantifying uncertainty in implicit 3D representations

© 2021 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting /republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other worksNeural Radiance Fields (NeRF) has become a popular framework for learning implicit 3D representations and addressing different tasks such as novel-view synthesis or depth-map estimation. However, in downstream applications where decisions need to be made based on automatic predictions, it is critical to leverage the confidence associated with the model estimations. Whereas uncertainty quantification is a long-standing problem in Machine Learning, it has been largely overlooked in the recent NeRF literature. In this context, we propose Stochastic Neural Radiance Fields (S-NeRF), a generalization of standard NeRF that learns a probability distribution over all the possible radiance fields modeling the scene. This distribution allows to quantify the uncertainty associated with the scene information provided by the model. S-NeRF optimization is posed as a Bayesian learning problem that is efficiently addressed using the Variational Inference framework. Exhaustive experiments over benchmark datasets demonstrate that S-NeRF is able to provide more reliable predictions and confidence values than generic approaches previously proposed for uncertainty estimation in other domains.This work is supported partly by the Chinese Scholarship Council (CSC) under grant (201906120031), by the Spanish government under project MoHuCo PID2020- 120049RB-I00, the ERA-Net Chistera project IPALM PCI2019-103386 and María de Maeztu Seal of Excellence MDM-2016-0656. Adria Ruiz acknowledges financial support from MICINN through the program Juan de la Cierva.Peer ReviewedPostprint (author's final draft

Towards Balanced Active Learning for Multimodal Classification

Training multimodal networks requires a vast amount of data due to their larger parameter space compared to unimodal networks. Active learning is a widely used technique for reducing data annotation costs by selecting only those samples that could contribute to improving model performance. However, current active learning strategies are mostly designed for unimodal tasks, and when applied to multimodal data, they often result in biased sample selection from the dominant modality. This unfairness hinders balanced multimodal learning, which is crucial for achieving optimal performance. To address this issue, we propose three guidelines for designing a more balanced multimodal active learning strategy. Following these guidelines, a novel approach is proposed to achieve more fair data selection by modulating the gradient embedding with the dominance degree among modalities. Our studies demonstrate that the proposed method achieves more balanced multimodal learning by avoiding greedy sample selection from the dominant modality. Our approach outperforms existing active learning strategies on a variety of multimodal classification tasks. Overall, our work highlights the importance of balancing sample selection in multimodal active learning and provides a practical solution for achieving more balanced active learning for multimodal classification.Comment: 12 pages, accepted by ACMMM 202

The role of preoperative pulmonary function tests in the surgical treatment of extremely severe scoliosis

Abstract Background The patients with extremely severe spinal deformity are commonly considered high-risk candidates for surgical treatment because of their underlying lung disease. Currently, little has been reported about the postoperative pulmonary complication events in this population. This retrospective study sought to evaluate preoperative pulmonary function tests in the surgical treatment of extremely severe scoliosis. Methods Preoperative forced vital capacity (FVC), FVC ratio, forced expiratory volume at the end of the first second (FEV1), FEV1 ratio, peak expiratory flow (PEF), and PEF ratio were performed and evaluated on 60 patients with extremely severe scoliosis (coronary main Cobb angle ≥100°). Results Among the 60 patients, 11 (18.3%), 13 (21.7%), and 22 (36.7%) had severe, moderate, and mild pulmonary dysfunction, respectively. Compared with the moderate and mild scoliosis groups, significant differences were observed in Cobb, FVC, FVC ratio, FEV1, FEV1 ratio, and PEF ratio in the extremely severe scoliosis group. Various postoperative pulmonary complications occurred in nine cases (15%). Patients with severe or moderate dysfunction as measured by the FVC ratio had a higher incidence of postoperative pulmonary complications. A transthoracic procedure was not related to postoperative pulmonary complications, but thoracoplasty significantly increased the incidence of postoperative pulmonary complications (P < 0.001, OR = 20, 95% CI = 3.45–115.97). Discussion Pulmonary function was impaired in extremely severe scoliosis. Patients with severe restrictive pulmonary dysfunction had a higher incidence of postoperative pulmonary complications. Thoracoplasty was an important risk factor in the prediction of postoperative pulmonary complications

The changes of the interspace angle after anterior correction and instrumentation in adolescent idiopathic scoliosis patients

<p>Abstract</p> <p>Background</p> <p>In idiopathic scoliosis patients, after anterior spinal fusion and instrumentation, the discs (interspace angle) between the lowest instrumented vertebra (LIV) and the next caudal vertebra became more wedged. We reviewed these patients and analyzed the changes of the angle.</p> <p>Methods</p> <p>By reviewing the medical records and roentgenograms of adolescent idiopathic scoliosis patients underwent anterior spinal fusion and instrumentation, Cobb angle of the curve, correction rate, coronal balance, LIV rotation, interspace angle were measured and analyzed.</p> <p>Results</p> <p>There were total 30 patients included. The mean coronal Cobb angle of the main curve (thoracolumbar/lumbar curve) before and after surgery were 48.9° and 11.7°, respectively, with an average correction rate of 76.1%. The average rotation of LIV before surgery was 2.1 degree, and was improved to 1.2 degree after surgery. The interspace angle before surgery, on convex side-bending films, after surgery, at final follow up were 3.2°, -2.3°, 1.8° and 4.9°, respectively. The difference between the interspace angle after surgery and that preoperatively was not significant (P = 0.261), while the interspace angle at final follow-up became larger than that after surgery, and the difference was significant(P = 0.012). The interspace angle after surgery was correlated with that on convex side-bending films (r = 0.418, P = 0.022), and the interspace angle at final follow-up was correlated with that after surgery (r = 0.625, P = 0.000). There was significant correlation between the loss of the interspace angle and the loss of coronal Cobb angle of the main curve during follow-up(r = 0.483, P = 0.007).</p> <p>Conclusion</p> <p>The interspace angle could be improved after anterior correction and instrumentation surgery, but it became larger during follow-up. The loss of the interspace angle was correlated with the loss of coronal Cobb angle of the main curve during follow-up.</p

Does Abnormal Preoperative Coagulation Status Lead to More Perioperative Blood Loss in Spinal Deformity Correction Surgery?

This study aims to analyze the potential association between the preoperative coagulation status and perioperative blood loss in spinal deformity correction surgery. The preoperative coagulation status and estimated blood loss (EBL) during operation, postoperative wound drainage, and allogeneic transfusion during and after operation were recorded and analyzed. Among the 164 patients, 26 had a longer prothrombin time (PT), 13 had a lower fibrinogen level, 55 had a longer activated partial thromboplastin time (APTT), and 2 had a longer thrombin time (TT), and the platelet count (PLT) was all normal or higher than the normal level. The mean EBL per surgical level was 77.8 ml (range, 22–267 ml), and the mean drainage per surgical level was 52.7 ml (range, 7–168 ml). Fifty-five patients and 12 patients underwent allogeneic transfusion during and after the operation, respectively. The differences in EBL per surgical level, mean drainage per surgical level, the occurrences of allogeneic transfusion during and after operation between the patients with a longer PT, lower fibrinogen level, longer APTT or longer TT, and the normal controls were not significant (all P’s &gt; 0.05). The Spearman correlation analysis showed that there was no correlation between PT, fibrinogen, APTT, TT or PLT with EBL per surgical level, mean drainage per surgical level, or allogeneic transfusion during and after the operation (all P’s &gt; 0.05). The abnormal preoperative coagulation status but not hemophilia does not lead to more perioperative blood loss or a higher rate of perioperative allogeneic transfusion in spinal deformity correction surgery