RAI-Net: Range-Adaptive LiDAR Point Cloud Frame Interpolation Network

LiDAR point cloud frame interpolation, which synthesizes the intermediate frame between the captured frames, has emerged as an important issue for many applications. Especially for reducing the amounts of point cloud transmission, it is by predicting the intermediate frame based on the reference frames to upsample data to high frame rate ones. However, due to high-dimensional and sparse characteristics of point clouds, it is more difficult to predict the intermediate frame for LiDAR point clouds than videos. In this paper, we propose a novel LiDAR point cloud frame interpolation method, which exploits range images (RIs) as an intermediate representation with CNNs to conduct the frame interpolation process. Considering the inherited characteristics of RIs differ from that of color images, we introduce spatially adaptive convolutions to extract range features adaptively, while a high-efficient flow estimation method is presented to generate optical flows. The proposed model then warps the input frames and range features, based on the optical flows to synthesize the interpolated frame. Extensive experiments on the KITTI dataset have clearly demonstrated that our method consistently achieves superior frame interpolation results with better perceptual quality to that of using state-of-the-art video frame interpolation methods. The proposed method could be integrated into any LiDAR point cloud compression systems for inter prediction.Comment: Accepted by the IEEE International Symposium on Broadband Multimedia Systems and Broadcasting 202

An unsupervised optical flow estimation for lidar image sequences

In recent years, the LiDAR images, as a 2D compact representation of 3D LiDAR point clouds, are widely applied in various tasks, e.g., 3D semantic segmentation, LiDAR point cloud compression (PCC). Among these works, the optical flow estimation for LiDAR image sequences has become a key issue, especially for the motion estimation of the inter prediction in PCC. However, the existing optical flow estimation models are likely to be unreliable for LiDAR images. In this work, we first propose a light-weight flow estimation model for LiDAR image sequences. The key novelty of our method lies in two aspects. One is that for the different characteristics (with the spatial-variation feature distribution) of the LiDAR images w.r.t. the normal color images, we introduce the attention mechanism into our model to improve the quality of the estimated flow. The other one is that to tackle the lack of large-scale LiDAR-image annotations, we present an unsupervised method, which directly minimizes the inconsistency between the reference image and the reconstructed image based on the estimated optical flow. Extensive experimental results have shown that our proposed model outperforms other mainstream models on the KITTI dataset, with much fewer parameters

Deep inter prediction via reference frame interpolation for blurry video coding

In High Efficiency Video Coding (HEVC), inter prediction is an important module for removing temporal redundancy. The accuracy of inter prediction is much affected by the similarity between the current and reference frames. However, for blurry videos, the performance of inter coding will be degraded by varying motion blur, which is derived from camera shake or the acceleration of objects in the scene. To address this problem, we propose to synthesize additional reference frame via the frame interpolation network. The synthesized reference frame is added into reference picture lists to supply more credible reference candidate, and the searching mechanism for motion candidates is changed accordingly. In addition, to make our interpolation network more robust to various inputs with different compression artifacts, we establish a new blurry video database to train our network. With the well-trained frame interpolation network, compared with the reference software HM-16.9, the proposed method achieves on average 1.55% BD-rate reduction under random access (RA) configuration for blurry videos, and also obtains on average 0.75% BD-rate reduction for common test sequences

Thermal Deformation Measurement of Aerospace Honeycomb Panel Based on Fusion of 3D-Digital Image Correlation and Finite Element Method

Aiming to solve the problem of the high-precision deformation measurement of large-scale satellite structures in manufacturing and testing environments, this paper proposes a measurement method based on the idea of fusing actual measurements with finite element analysis. The digital image correlation (DIC) method is used to obtain the high-precision deformation of the honeycomb panel, and the finite element method (FEM) model is introduced to remove the limitations of existing pure measurement methods. Data fusion based on a machine learning neural network is proposed to fuse high-precision deformation and physical information such as temperature to conduct multi-level training on FEM parameters. Through an interpolation of the analysis and calculation results after training, not only can the accuracy of the finite element be improved, but difference and extrapolation of the digital image correlation measurement results can be performed. In the experiments, the satellite on-orbit temperature data are substituted into the modified finite element model. The testing results shows that the prediction accuracy of the model under different temperature loads can be controlled within 10 μm under an 840 mm × 640 mm scale. A high predictive accuracy can be achieved for the revised model for the complete deformation of large structural sections

PAID study design on the role of PKC activation in immune/inflammation-related depression: a randomised placebo-controlled trial protocol

Background Inflammation that is mediated by microglia activation plays an important role in the pathogenesis of depression. Microglia activation can lead to an increase in the levels of proinflammatory cytokines, including TNF-α, which leads to neuronal apoptosis in the specific neural circuits of some brain regions, abnormal cognition and treatment-resistant depression (TRD). Protein kinase C (PKC) is a key regulator of the microglia activation process. We assume that the abnormality in PKC might result in abnormal microglia activation, neuronal apoptosis, significant changes in emotional and cognitive neural circuits, and TRD. In the current study, we plan to target at the PKC signal pathway to improve the TRD treatment outcome.Methods and analysis This is a 12-week, ongoing, randomised, placebo-controlled trial. Patients with TRD (N=180) were recruited from Shanghai Mental Health Center, Shanghai Jiao Tong University. Healthy control volunteers (N=60) were recruited by advertisement. Patients with TRD were randomly assigned to ‘escitalopram+golimumab (TNF-α inhibitor)’, ‘escitalopram+calcium tablet+vitamin D (PKC activator)’ or ‘escitalopram+placebo’ groups. We define the primary outcome as changes in the 17-item Hamilton Depression Rating Scale (HAMD-17). The secondary outcome is defined as changes in anti-inflammatory effects, cognitive function and quality of life.Discussion This study might be the first randomised, placebo-controlled trial to target at the PKC signal pathway in patients with TRD. Our study might help to propose individualised treatment strategies for depression.Trial registration number The trial protocol is registered with ClinicalTrials.gov under protocol ID 81930033 and ClinicalTrials.gov ID NCT04156425