Revisiting Rolling Shutter Bundle Adjustment: Toward Accurate and Fast Solution

We propose a robust and fast bundle adjustment solution that estimates the 6-DoF pose of the camera and the geometry of the environment based on measurements from a rolling shutter (RS) camera. This tackles the challenges in the existing works, namely relying on additional sensors, high frame rate video as input, restrictive assumptions on camera motion, readout direction, and poor efficiency. To this end, we first investigate the influence of normalization to the image point on RSBA performance and show its better approximation in modelling the real 6-DoF camera motion. Then we present a novel analytical model for the visual residual covariance, which can be used to standardize the reprojection error during the optimization, consequently improving the overall accuracy. More importantly, the combination of normalization and covariance standardization weighting in RSBA (NW-RSBA) can avoid common planar degeneracy without needing to constrain the filming manner. Besides, we propose an acceleration strategy for NW-RSBA based on the sparsity of its Jacobian matrix and Schur complement. The extensive synthetic and real data experiments verify the effectiveness and efficiency of the proposed solution over the state-of-the-art works. We also demonstrate the proposed method can be easily implemented and plug-in famous GSSfM and GSSLAM systems as completed RSSfM and RSSLAM solutions

Towards Nonlinear-Motion-Aware and Occlusion-Robust Rolling Shutter Correction

This paper addresses the problem of rolling shutter correction in complex nonlinear and dynamic scenes with extreme occlusion. Existing methods suffer from two main drawbacks. Firstly, they face challenges in estimating the accurate correction field due to the uniform velocity assumption, leading to significant image correction errors under complex motion. Secondly, the drastic occlusion in dynamic scenes prevents current solutions from achieving better image quality because of the inherent difficulties in aligning and aggregating multiple frames. To tackle these challenges, we model the curvilinear trajectory of pixels analytically and propose a geometry-based Quadratic Rolling Shutter (QRS) motion solver, which precisely estimates the high-order correction field of individual pixels. Besides, to reconstruct high-quality occlusion frames in dynamic scenes, we present a 3D video architecture that effectively Aligns and Aggregates multi-frame context, namely, RSA2-Net. We evaluate our method across a broad range of cameras and video sequences, demonstrating its significant superiority. Specifically, our method surpasses the state-of-the-art by +4.98, +0.77, and +4.33 of PSNR on Carla-RS, Fastec-RS, and BS-RSC datasets, respectively. Code is available at https://github.com/DelinQu/qrsc.Comment: accepted at ICCV 202

Fast Rolling Shutter Correction in the Wild

This paper addresses the problem of rolling shutter correction (RSC) in uncalibrated videos. Existing works remove rolling shutter (RS) distortion by explicitly computing the camera motion and depth as intermediate products, followed by motion compensation. In contrast, we first show that each distorted pixel can be implicitly rectified back to the corresponding global shutter (GS) projection by rescaling its optical flow. Such a point-wise RSC is feasible with both perspective, and non-perspective cases without the pre-knowledge of the camera used. Besides, it allows a pixel-wise varying RSC framework called DRSC that handles locally varying distortion caused by various sources, such as camera motion, moving objects, and depth variation in a scene. More importantly, our approach is an efficient CPU-based solution that enables undistorting RS video in real-time (40fps for 480p). We evaluate our approach across a broad range of cameras and video sequences, including fast motion, dynamic scenes, and non-perspective lenses, demonstrating the superiority of our proposed approach over state-of-the-art methods in both effectiveness and efficiency. We also evaluated the ability of the RSC results to serve for downstream 3D analysis, such as visual odometry and structure-from-motion, which verifies preference for the output of our algorithm over other existing RSC methods

Short-term intersection traffic flow forecasting

The intersection is a bottleneck in an urban roadway network. As traffic demand increases, there is a growing congestion problem at urban intersections. Short-term traffic flow forecasting is crucial for advanced trip planning and traffic management. However, there are only a handful of existing models for forecasting intersection traffic flow. In addition, previous short-term traffic flow forecasting models usually were for predicting roadway conditions in a very short period, such as one minute or five minutes, which is often too late given that a driver may well be approaching the bottleneck already. Being able to accurately predict traffic congestions in about half-hour advance is very critical for advanced trip planning and traffic management. To fill this gap, this research develops a two-layer stacking model for intersection short-term traffic flow forecasting by integrating the K-nearest neighbor (KNN) and Elman Neural Network modeling methods. It was developed using the 24-h cycle by cycle traffic data collected at a signalized intersection in Jinan, China. The developed model is evaluated by applying it to the same intersection for forecasting the short-term traffic conditions in a different set of days. The prediction performance of this model was compared with four other models developed using some existing non-parametric modeling and machine learning methods, including clustering, backpropagation (BP) neural network, KNN, and Elman Neural Network. The results of this study indicate that the proposed model outperforms other existing models in terms of its prediction accuracy

Clinical updates on gliomas and implications of the 5th edition of the WHO classification of central nervous system tumors

BackgroundThe 5th edition of the World Health Organization (WHO) classification of central nervous system tumors incorporated specific molecular alterations into the categorization of gliomas. The major revision of the classification scheme effectuates significant changes in the diagnosis and management of glioma. This study aimed to depict the clinical, molecular, and prognostic characteristics of glioma and its subtypes according to the current WHO classification.MethodsPatients who underwent surgery for glioma at Peking Union Medical College Hospital during 11 years were re-examined for tumor genetic alterations using next-generation sequencing, polymerase chain reaction-based assay, and fluorescence in situ hybridization methods and enrolled in the analysis.ResultsThe enrolled 452 gliomas were reclassified into adult-type diffuse glioma (ntotal=373; astrocytoma, n=78; oligodendroglioma, n=104; glioblastoma, n=191), pediatric-type diffuse glioma (ntotal=23; low-grade, n=8; high-grade, n=15), circumscribed astrocytic glioma (n=20), and glioneuronal and neuronal tumor (n=36). The composition, definition, and incidence of adult- and pediatric-type gliomas changed significantly between the 4th and the 5th editions of the classification. The clinical, radiological, molecular, and survival characteristics of each subtype of glioma were identified. Alterations in CDK4/6, CIC, FGFR2/3/4, FUBP1, KIT, MET, NF1, PEG3, RB1, and NTRK2 were additional factors correlated with the survival of different subtypes of gliomas.ConclusionsThe updated WHO classification based on histology and molecular alterations has updated our understanding of the clinical, radiological, molecular, survival, and prognostic characteristics of varied subtypes of gliomas and provided accurate guidance for diagnosis and potential prognosis for patients

Novel insight into histological and molecular astrocytoma, IDH‐mutant, Grade 4 by the updated WHO classification of central nervous system tumors

Abstract Background The latest fifth edition of the World Health Organization (WHO) classification of the central nervous system (CNS) tumors (WHO CNS 5 classification) released in 2021 defined astrocytoma, IDH‐mutant, Grade 4. However, the understanding of this subtype is still limited. We conducted this study to describe the features of astrocytoma, IDH‐mutant, Grade 4 and explored the similarities and differences between histological and molecular subtypes. Methods Patients who underwent surgery from January 2011 to January 2022, classified as astrocytoma, IDH‐mutant, Grade 4 were included in this study. Clinical, radiological, histopathological, molecular pathological, and survival data were collected for analysis. Results Altogether 33 patients with astrocytoma, IDH‐mutant, Grade 4 were selected, including 20 with histological and 13 with molecular WHO Grade 4 astrocytoma. Tumor enhancement, intratumoral‐necrosis like presentation, larger peritumoral edema, and more explicit tumor margins were frequently observed in histological WHO Grade 4 astrocytoma. Additionally, molecular WHO Grade 4 astrocytoma showed a tendency for relatively longer overall survival, while a statistical significance was not reached (47 vs. 25 months, p = 0.22). TP53, CDK6, and PIK3CA alteration was commonly observed, while PIK3R1 (p = 0.033), Notch1 (p = 0.027), and Mycn (p = 0.027) alterations may affect the overall survival of molecular WHO Grade 4 astrocytomas. Conclusions Our study scrutinized IDH‐mutant, Grade 4 astrocytoma. Therefore, further classification should be considered as the prognosis varied between histological and molecular WHO Grade 4 astrocytomas. Notably, therapies aiming at PIK3R1, Notch 1, and Mycn may be beneficial