USB-NeRF: Unrolling Shutter Bundle Adjusted Neural Radiance Fields

Neural Radiance Fields (NeRF) has received much attention recently due to its impressive capability to represent 3D scene and synthesize novel view images. Existing works usually assume that the input images are captured by a global shutter camera. Thus, rolling shutter (RS) images cannot be trivially applied to an off-the-shelf NeRF algorithm for novel view synthesis. Rolling shutter effect would also affect the accuracy of the camera pose estimation (e.g. via COLMAP), which further prevents the success of NeRF algorithm with RS images. In this paper, we propose Unrolling Shutter Bundle Adjusted Neural Radiance Fields (USB-NeRF). USB-NeRF is able to correct rolling shutter distortions and recover accurate camera motion trajectory simultaneously under the framework of NeRF, by modeling the physical image formation process of a RS camera. Experimental results demonstrate that USB-NeRF achieves better performance compared to prior works, in terms of RS effect removal, novel view image synthesis as well as camera motion estimation. Furthermore, our algorithm can also be used to recover high-fidelity high frame-rate global shutter video from a sequence of RS images

Revisiting Rolling Shutter Bundle Adjustment: Toward Accurate and Fast Solution

We propose a robust and fast bundle adjustment solution that estimates the 6-DoF pose of the camera and the geometry of the environment based on measurements from a rolling shutter (RS) camera. This tackles the challenges in the existing works, namely relying on additional sensors, high frame rate video as input, restrictive assumptions on camera motion, readout direction, and poor efficiency. To this end, we first investigate the influence of normalization to the image point on RSBA performance and show its better approximation in modelling the real 6-DoF camera motion. Then we present a novel analytical model for the visual residual covariance, which can be used to standardize the reprojection error during the optimization, consequently improving the overall accuracy. More importantly, the combination of normalization and covariance standardization weighting in RSBA (NW-RSBA) can avoid common planar degeneracy without needing to constrain the filming manner. Besides, we propose an acceleration strategy for NW-RSBA based on the sparsity of its Jacobian matrix and Schur complement. The extensive synthetic and real data experiments verify the effectiveness and efficiency of the proposed solution over the state-of-the-art works. We also demonstrate the proposed method can be easily implemented and plug-in famous GSSfM and GSSLAM systems as completed RSSfM and RSSLAM solutions

Fast Rolling Shutter Correction in the Wild

This paper addresses the problem of rolling shutter correction (RSC) in uncalibrated videos. Existing works remove rolling shutter (RS) distortion by explicitly computing the camera motion and depth as intermediate products, followed by motion compensation. In contrast, we first show that each distorted pixel can be implicitly rectified back to the corresponding global shutter (GS) projection by rescaling its optical flow. Such a point-wise RSC is feasible with both perspective, and non-perspective cases without the pre-knowledge of the camera used. Besides, it allows a pixel-wise varying RSC framework called DRSC that handles locally varying distortion caused by various sources, such as camera motion, moving objects, and depth variation in a scene. More importantly, our approach is an efficient CPU-based solution that enables undistorting RS video in real-time (40fps for 480p). We evaluate our approach across a broad range of cameras and video sequences, including fast motion, dynamic scenes, and non-perspective lenses, demonstrating the superiority of our proposed approach over state-of-the-art methods in both effectiveness and efficiency. We also evaluated the ability of the RSC results to serve for downstream 3D analysis, such as visual odometry and structure-from-motion, which verifies preference for the output of our algorithm over other existing RSC methods

Monoamine oxidase A mediates prostate tumorigenesis and cancer metastasis

Tumors from patients with high-grade aggressive prostate cancer (PCa) exhibit increased expression of monoamine oxidase A (MAOA), a mitochondrial enzyme that degrades monoamine neurotransmitters and dietary amines. Despite the association between MAOA and aggressive PCa, it is unclear how MAOA promotes PCa progression. Here, we found that MAOA functions to induce epithelial-to-mesenchymal transition (EMT) and stabilize the transcription factor HIF1α, which mediates hypoxia through an elevation of ROS, thus enhancing growth, invasiveness, and metastasis of PCa cells. Knockdown and overexpression of MAOA in human PCa cell lines indicated that MAOA induces EMT through activation of VEGF and its coreceptor neuropilin-1. MAOA-dependent activation of neuropilin-1 promoted AKT/FOXO1/TWIST1 signaling, allowing FOXO1 binding at theTWIST1promoter. Importantly, the MAOA-dependent HIF1α/VEGF-A/FOXO1/TWIST1 pathway was activated in high-grade PCa specimens, and knockdown of MAOA reduced or even eliminated prostate tumor growth and metastasis in PCa xenograft mouse models. Pharmacological inhibition of MAOA activity also reduced PCa xenograft growth in mice. Moreover, high MAOA expression in PCa tissues correlated with worse clinical outcomes in PCa patients. These findings collectively characterize the contribution of MAOA in PCa pathogenesis and suggest that MAOA has potential as a therapeutic target in PCa