Look, Listen and Learn - A Multimodal LSTM for Speaker Identification

Speaker identification refers to the task of localizing the face of a person who has the same identity as the ongoing voice in a video. This task not only requires collective perception over both visual and auditory signals, the robustness to handle severe quality degradations and unconstrained content variations are also indispensable. In this paper, we describe a novel multimodal Long Short-Term Memory (LSTM) architecture which seamlessly unifies both visual and auditory modalities from the beginning of each sequence input. The key idea is to extend the conventional LSTM by not only sharing weights across time steps, but also sharing weights across modalities. We show that modeling the temporal dependency across face and voice can significantly improve the robustness to content quality degradations and variations. We also found that our multimodal LSTM is robustness to distractors, namely the non-speaking identities. We applied our multimodal LSTM to The Big Bang Theory dataset and showed that our system outperforms the state-of-the-art systems in speaker identification with lower false alarm rate and higher recognition accuracy.Comment: The 30th AAAI Conference on Artificial Intelligence (AAAI-16

Dynamic analysis of Th1/Th2 cytokine concentration during antiretroviral therapy of HIV-1/HCV co-infected Patients

<p>Abstract</p> <p>Background</p> <p>Co-infection with hepatitis C (HCV) is very common in human immunodeficiency virus 1 (HIV-1) infected patients. Although HIV co-infection clearly accelerates progression of HCV-related fibrosis and liver disease, controversy remains as to the impact of HCV on HIV disease progression in co-infected patients. HIV can cause immune dysfunction, in which the regulatory function of T helper (Th) cells is very essential. Moreover, cytokines derived from Th cells play a prominent role in viral infection. Investigating the functional changes of Th1 and Th2 cells in cytokine level can improve the understanding of the effect of co-infected HCV on HIV infection.</p> <p>Methods</p> <p>In this study, we measured the baseline Th1/Th2 cytokine concentration in sera by using flow cytometry in HIV/HCV co-infection, HIV mono-infection, HCV mono-infection, and healthy control group, as well as the dynamic changes of these cytokine levels after receiving highly active antiretroviral therapy (HAART).</p> <p>Results</p> <p>The ratio of Th1 and Th2 cytokine concentration in HIV/HCV co-infection was higher than HCV mono-infection and healthy control group, while lower than HIV mono-infection group. After HAART was initiated, the Th1/Th2 ratio of HIV/HCV co-infection group decreased to the same level of healthy control, while HIV mono-infection group was still higher than the control group.</p> <p>Conclusions</p> <p>There was no significant evidence showing co-infected with HCV had negative effect on HIV related diseases. However, co-infected with HCV can decrease Th1/Th2 ratio by affecting Th1 cytokine level, especially the secretion of IFN-γ. With the initiation of HAART, Th1 and Th2 cytokine levels were progressively reduced. HIV was the main stimulating factor of T cells in HIV/HCV co-infection group.</p

RGBT Salient Object Detection: A Large-scale Dataset and Benchmark

Salient object detection in complex scenes and environments is a challenging research topic. Most works focus on RGB-based salient object detection, which limits its performance of real-life applications when confronted with adverse conditions such as dark environments and complex backgrounds. Taking advantage of RGB and thermal infrared images becomes a new research direction for detecting salient object in complex scenes recently, as thermal infrared spectrum imaging provides the complementary information and has been applied to many computer vision tasks. However, current research for RGBT salient object detection is limited by the lack of a large-scale dataset and comprehensive benchmark. This work contributes such a RGBT image dataset named VT5000, including 5000 spatially aligned RGBT image pairs with ground truth annotations. VT5000 has 11 challenges collected in different scenes and environments for exploring the robustness of algorithms. With this dataset, we propose a powerful baseline approach, which extracts multi-level features within each modality and aggregates these features of all modalities with the attention mechanism, for accurate RGBT salient object detection. Extensive experiments show that the proposed baseline approach outperforms the state-of-the-art methods on VT5000 dataset and other two public datasets. In addition, we carry out a comprehensive analysis of different algorithms of RGBT salient object detection on VT5000 dataset, and then make several valuable conclusions and provide some potential research directions for RGBT salient object detection.Comment: 12 pages, 10 figures https://github.com/lz118/RGBT-Salient-Object-Detectio

Effects of oxytetracycline and sulfachloropyridazine residues on the reductive activity of Shewanella decolorationis S12

Effects of oxytetracycline (OTC) and sulfachloropyridazine (SCP), two of the widely used antibiotics in livestock production, on beneficial environmental microorganisms were studied. Shewanella decolorationis S12 was selected as the target bacteria for the role in reduction of Fe(III) and dye under anaerobic conditions. The results showed that the antibiotics significantly inhibited Fe(III) reduction and dye decoloration in the reduction system. The rates of Fe(II) formed (−r) were 3.6 and 0.2 mg/L/day for the OTC concentrations of 0−1 mg/L and 1−50 mg/L, respectively, with 1 mg/L as the turning point of the inhibition effect. The turning point of inhibition effect was much higher for SCP treatments, at 4 mg/L. The results also showed higher production values for adsorbed Fe(II) than soluble Fe(II) in OTC treatments, but the reverse occurred in the SCP treatments. The difference between the treatments could be due to higher sorption coefficients of OTC as compared to SCP. Transmission electron micrographs showed changes in cell structures of S. decolorationis S12 grown in medium with OTC. Detached cell walls and large vacuoles in internal cell contents were found in OTC-treated cells. The results of the present study indicated that the inhibition of antibiotic on the reduction activity of S. decolorationis S12 may be due to a decrease in live S. decolorationis S12 population and/or damages of their cell structure in this reduction system

Efficacy and safety of neoadjuvant immunotherapy in resectable esophageal or gastroesophageal junction carcinoma: A pooled analysis of prospective clinical trials

Neoadjuvant chemoradiotherapy (NCRT) plus radical esophagectomy is currently the standard treatment for resectable esophageal or gastroesophageal junction (GEJ) carcinoma. The aim of this study is to evaluate the efficacy and safety of neoadjuvant immunotherapy in resectable esophageal or GEJ carcinoma. Prospective clinical trials investigating efficacy and/or safety of neoadjuvant immunotherapy with immune checkpoint inhibitors (ICIs) followed by radical esophagectomy in patients with newly diagnosed resectable esophageal or GEJ carcinoma were identified through literature search. Quality assessment was performed by using the Newcastle–Ottawa scale. Preliminary treatment outcomes of pathologically complete response (pCR, ypT0N0) and grade 3-4 adverse effects (AEs) were pooled together and then compared with standard NCRT of the historical control CROSS study by Chi-square (χ2) test. A two-sided P value &lt; 0.05 was considered statistically significant. A total of 17 eligible non-randomized trials with 455 participants were included into analysis. The most common primary endpoint was pCR (n = 7, 41%), and the median sample size and follow-up period was 23 patients and 7.9 months, respectively. For patients receiving neoadjuvant immunotherapy, the overall pCR, R0 resection, and grade 3-4 AE rates were 33.2%, 95.5%, and 35.1%, respectively. For esophageal squamous cell carcinoma (ESCC) and adenocarcinoma (EAC), neoadjuvant immunochemoradiotherapy showed no significant improvement in pCR rate than NCRT (ESCC, 50% vs 48.7%, P = 0.9; EAC, 32.6% vs 23.1%, P = 0.22). Grade 3-4 AEs were the most common in patients with neoadjuvant immunochemoradiotherapy, significantly higher than immunochemotherapy (46.7% vs 32.8%, P = 0.04) and NCRT (46.7% vs 18.1%, P &lt; 0.0001). In conclusion, for patients with resectable esophageal or GEJ carcinoma, the addition of ICIs to standard NCRT could not improve pCR rate in both ESCC and EAC, but significantly increased the risk of severe AEs. Large-scale phase 3 randomized trials were urgently needed to further confirm the survival benefit and safety profile of neoadjuvant immunotherapy

Direct and indirect effects of climatic variations on the interannual variability in net ecosystem exchange across terrestrial ecosystems

Climatic variables not only directly affect the interannual variability (IAV) in net ecosystem exchange of CO2 (NEE) but also indirectly drive it by changing the physiological parameters. Identifying these direct and indirect paths can reveal the underlying mechanisms of carbon (C) dynamics. In this study, we applied a path analysis using flux data from 65 sites to quantify the direct and indirect climatic effects on IAV in NEE and to evaluate the potential relationships among the climatic variables and physiological parameters that represent physiology and phenology of ecosystems. We found that the maximum photosynthetic rate was the most important factor for the IAV in gross primary productivity (GPP), which was mainly induced by the variation in vapour pressure deficit. For ecosystem respiration (RE), the most important drivers were GPP and the reference respiratory rate. The biome type regulated the direct and indirect paths, with distinctive differences between forests and non-forests, evergreen needleleaf forests and deciduous broadleaf forests, and between grasslands and croplands. Different paths were also found among wet, moist and dry ecosystems. However, the climatic variables can only partly explain the IAV in physiological parameters, suggesting that the latter may also result from other biotic and disturbance factors. In addition, the climatic variables related to NEE were not necessarily the same as those related to GPP and RE, indicating the emerging difficulty encountered when studying the IAV in NEE. Overall, our results highlight the contribution of certain physiological parameters to the IAV in C fluxes and the importance of biome type and multi-year water conditions, which should receive more attention in future experimental and modelling research

Variance estimation for statistics computed from single recurrent event processes

This article is concerned with variance estimation for statistics that are computed from single recurrent event processes. Such statistics are important in diagnosis for each individual recurrent event process. The proposed method only assumes a semiparametric form for the first-order structure of the processes but not for the second-order (i.e., dependence) structure. The new variance estimator is shown to be consistent for the target parameter under very mild conditions. The estimator can be used in many applications in semiparametric rate regression analysis of recurrent event data such as outlier detection, residual diagnosis, as well as robust regression. A simulation study and application to two real data examples are used to demonstrate the use of the proposed method