From Deterministic to Generative: Multi-Modal Stochastic RNNs for Video Captioning

Video captioning in essential is a complex natural process, which is affected by various uncertainties stemming from video content, subjective judgment, etc. In this paper we build on the recent progress in using encoder-decoder framework for video captioning and address what we find to be a critical deficiency of the existing methods, that most of the decoders propagate deterministic hidden states. Such complex uncertainty cannot be modeled efficiently by the deterministic models. In this paper, we propose a generative approach, referred to as multi-modal stochastic RNNs networks (MS-RNN), which models the uncertainty observed in the data using latent stochastic variables. Therefore, MS-RNN can improve the performance of video captioning, and generate multiple sentences to describe a video considering different random factors. Specifically, a multi-modal LSTM (M-LSTM) is first proposed to interact with both visual and textual features to capture a high-level representation. Then, a backward stochastic LSTM (S-LSTM) is proposed to support uncertainty propagation by introducing latent variables. Experimental results on the challenging datasets MSVD and MSR-VTT show that our proposed MS-RNN approach outperforms the state-of-the-art video captioning benchmarks

A multi-rule-based relative radiometric normalization for multi-sensor satellite images

Relative radiometric normalization (RRN) is a widely used method for enhancing the radiometric consistency among multi-temporal satellite images. Diverse satellite images enhance the information for observing the Earth’s surface and bring additional uncertainties in the applications using multi-sensor images, such as change detection, multi-temporal analysis, image fusion, etc. To address this challenge, we developed a multi-rule-based RRN method for multi-sensor satellite images, which involves the identification of spectral- and spatial-invariant pseudo-invariant features (PIFs) and a Partial least-squares (PLS) regression-based RRN modeling using neighboring target pixels around PIFs. The proposed RRN method was validated on four datasets and demonstrated excellent effectiveness in identifying high-quality PIFs with spectral- and spatial-invariant properties, estimating precise regression models, and enhancing the radiometric consistency of reference-target image pair. Our method outperformed six RRN methods and effectively processed well-registered medium- and high-resolution images from the same sensor. This letter highlights the potential of our method for generating more comparable bi-temporal multi-sensor images

A Spatiotemporal Volumetric Interpolation Network for 4D Dynamic Medical Image

Dynamic medical imaging is usually limited in application due to the large radiation doses and longer image scanning and reconstruction times. Existing methods attempt to reduce the dynamic sequence by interpolating the volumes between the acquired image volumes. However, these methods are limited to either 2D images and/or are unable to support large variations in the motion between the image volume sequences. In this paper, we present a spatiotemporal volumetric interpolation network (SVIN) designed for 4D dynamic medical images. SVIN introduces dual networks: first is the spatiotemporal motion network that leverages the 3D convolutional neural network (CNN) for unsupervised parametric volumetric registration to derive spatiotemporal motion field from two-image volumes; the second is the sequential volumetric interpolation network, which uses the derived motion field to interpolate image volumes, together with a new regression-based module to characterize the periodic motion cycles in functional organ structures. We also introduce an adaptive multi-scale architecture to capture the volumetric large anatomy motions. Experimental results demonstrated that our SVIN outperformed state-of-the-art temporal medical interpolation methods and natural video interpolation methods that have been extended to support volumetric images. Our ablation study further exemplified that our motion network was able to better represent the large functional motion compared with the state-of-the-art unsupervised medical registration methods.Comment: 10 pages, 8 figures, Conference on Computer Vision and Pattern Recognition (CVPR) 202

The Improvement of Hyperglycemia after RYGB Surgery in Diabetic Rats Is Related to Elevated Hypothalamus GLP-1 Receptor Expression

Objectives. This study aimed to explore the expression of GLP-1 receptor in hypothalamus and gastrointestinal tissues after Roux-en-Y gastric bypass (RYGB) surgery in diabetic rats. Methods. Male 12-week-old Wistar rats (control) and Goto-Kakizaki rats (diabetic) were randomly divided into two groups, respectively: control sham surgery group (C), control RYGB group (C + R), diabetic sham surgery group (D), and diabetic RYGB group (D + R). Body weight and blood glucose were monitored before and after surgery every week. Eight weeks after surgery, all rats were sacrificed and the serum fasting GLP-1 concentrations were measured by ELISA. GLP-1R and DPP-4 expression in hypothalamus and ileum were measured by RT-PCR. Results. The body weight and fasting/random blood glucose in the D + R group decreased significantly compared with the D group (P<0.05). Serum GLP-1 levels in diabetic rats treated with RYGB were higher than the corresponding sham surgery rats. The expression of GLP-1R of hypothalamus in RYGB-treated diabetic rats was significantly higher than those of the sham surgery diabetic rats and both control group rats (P<0.05). We found a negative correlation between hypothalamus GLP-1R mRNA and blood glucose level. No significant difference was seen in ileum GLP-1R and DPP-4 expression among all groups. Conclusions. RYGB efficiently promoted serum GLP-1 levels and the expression of GLP-1 receptor in the hypothalamus in diabetic rats. These data suggest that the hypothalamus GLP-1R may play an important role in the GLP-1 system for improving glucose homeostasis after reconstruction of the gastrointestinal tract

Effect of Patient Decision Aids in the Diagnosis and Treatment of Coronary Artery Disease：a Systematic Review

BackgroundPatient decision aid is recognized as an essential tool for shared-decision making. However, it is not clear that its role in shared-decision making in the diagnosis and treatment of coronary artery disease (CAD) .ObjectiveTo assess the effect of patient decision aids in the diagnosis and treatment of CAD using a systematic review.MethodsDatabases of PubMed, Web of Science, EMBase and The Cochrane Library were searched from inception to March 2021 for randomized trials assessing the effect of patient decision aids used in the diagnosis and treatment of CAD patients using a type of searching algorithm consisting of subject headings and free words. The Cochrane risk-of-bias tool for randomized trials (RoB2) was used for assessing risk of bias. Data were extracted, and effects of patient decision aids were summarized.ResultsA total of six randomized trials were included, and their qualities were moderate on the whole. Four were published within the past five years; five were conducted in the United States; three focus on the treatment of CAD and another three are about chest pain assessment due to suspected CAD. The effects of patient decision aids were summarized as follows: (1) With the support of a patient decision aid, patients obtained changes in their decision-making behaviors (two studies) , increased CAD-related knowledge (all studies) , reduced decisional conflicts (three studies) , and higher rate of attending decision-making (two studies) . (2) Most of the patient decision aids are web-based, and their contents mainly include information related to CAD, clarifying the pros and cons of treatment schemes for CAD, and personal risk assessments.ConclusionThe effects of patient decision aids are limited in the diagnosis and treatment of CAD, yet they have broad prospect in clinical practice. To promote their application in China, it is suggested to strengthen relevant trainings for clinicians to develop patient decision aids in line with features of Chinese culture and patients

Ginsenoside Rh1 Improves the Effect of Dexamethasone on Autoantibodies Production and Lymphoproliferation in MRL/lpr Mice

Ginsenoside Rh1 is able to upregulate glucocorticoid receptor (GR) level, suggesting Rh1 may improve glucocorticoid efficacy in hormone-dependent diseases. Therefore, we investigated whether Rh1 could enhance the effect of dexamethasone (Dex) in the treatment of MRL/lpr mice. MRL/lpr mice were treated with vehicle, Dex, Rh1, or Dex + Rh1 for 4 weeks. Dex significantly reduced the proteinuria and anti-dsDNA and anti-ANA autoantibodies. The levels of proteinuria and anti-dsDNA and anti-ANA autoantibodies were further decreased in Dex + Rh1 group. Dex, Rh1, or Dex + Rh1 did not alter the proportion of CD4+ splenic lymphocytes, whereas the proportion of CD8+ splenic lymphocytes was significantly increased in Dex and Dex + Rh1 groups. Dex + Rh1 significantly decreased the ratio of CD4+/CD8+ splenic lymphocytes compared with control. Con A-induced CD4+ splenic lymphocytes proliferation was increased in Dex-treated mice and was inhibited in Dex + Rh1-treated mice. Th1 cytokine IFN-γ mRNA was suppressed and Th2 cytokine IL-4 mRNA was increased by Dex. The effect of Dex on IFN-γ and IL-4 mRNA was enhanced by Rh1. In conclusion, our data suggest that Rh1 may enhance the effect of Dex in the treatment of MRL/lpr mice through regulating CD4+ T cells activation and Th1/Th2 balance