Understanding Convolution for Semantic Segmentation

Recent advances in deep learning, especially deep convolutional neural networks (CNNs), have led to significant improvement over previous semantic segmentation systems. Here we show how to improve pixel-wise semantic segmentation by manipulating convolution-related operations that are of both theoretical and practical value. First, we design dense upsampling convolution (DUC) to generate pixel-level prediction, which is able to capture and decode more detailed information that is generally missing in bilinear upsampling. Second, we propose a hybrid dilated convolution (HDC) framework in the encoding phase. This framework 1) effectively enlarges the receptive fields (RF) of the network to aggregate global information; 2) alleviates what we call the "gridding issue" caused by the standard dilated convolution operation. We evaluate our approaches thoroughly on the Cityscapes dataset, and achieve a state-of-art result of 80.1% mIOU in the test set at the time of submission. We also have achieved state-of-the-art overall on the KITTI road estimation benchmark and the PASCAL VOC2012 segmentation task. Our source code can be found at https://github.com/TuSimple/TuSimple-DUC .Comment: WACV 2018. Updated acknowledgements. Source code: https://github.com/TuSimple/TuSimple-DU

Spin Coherence and Spin Relaxation in Hybrid Organic-Inorganic Lead and Mixed Lead-Tin Perovskites

Metal halide perovskites make up a promising class of materials for semiconductor spintronics. Here we report a systematic investigation of coherent spin precession, spin dephasing and spin relaxation of electrons and holes in two hybrid organic-inorganic perovskites MA0.3FA0.7PbI3 and MA0.3FA0.7Pb0.5Sn0.5I3 using time-resolved Faraday rotation spectroscopy. With applied in-plane magnetic fields, we observe robust Larmor spin precession of electrons and holes that persists for hundreds of picoseconds. The spin dephasing and relaxation processes are likely to be sensitive to the defect levels. Temperature-dependent measurements give further insights into the spin relaxation channels. The extracted electron Land\'e g-factors (3.75 and 4.36) are the biggest among the reported values in inorganic or hybrid perovskites. Both the electron and hole g-factors shift dramatically with temperature, which we propose to originate from thermal lattice vibration effects on the band structure. These results lay the foundation for further design and use of lead- and tin-based perovskites for spintronic applications

Replay-enhanced Continual Reinforcement Learning

Replaying past experiences has proven to be a highly effective approach for averting catastrophic forgetting in supervised continual learning. However, some crucial factors are still largely ignored, making it vulnerable to serious failure, when used as a solution to forgetting in continual reinforcement learning, even in the context of perfect memory where all data of previous tasks are accessible in the current task. On the one hand, since most reinforcement learning algorithms are not invariant to the reward scale, the previously well-learned tasks (with high rewards) may appear to be more salient to the current learning process than the current task (with small initial rewards). This causes the agent to concentrate on those salient tasks at the expense of generality on the current task. On the other hand, offline learning on replayed tasks while learning a new task may induce a distributional shift between the dataset and the learned policy on old tasks, resulting in forgetting. In this paper, we introduce RECALL, a replay-enhanced method that greatly improves the plasticity of existing replay-based methods on new tasks while effectively avoiding the recurrence of catastrophic forgetting in continual reinforcement learning. RECALL leverages adaptive normalization on approximate targets and policy distillation on old tasks to enhance generality and stability, respectively. Extensive experiments on the Continual World benchmark show that RECALL performs significantly better than purely perfect memory replay, and achieves comparable or better overall performance against state-of-the-art continual learning methods.Comment: Accepted by Transactions on Machine Learning Research 202

Learning from Future: A Novel Self-Training Framework for Semantic Segmentation

Self-training has shown great potential in semi-supervised learning. Its core idea is to use the model learned on labeled data to generate pseudo-labels for unlabeled samples, and in turn teach itself. To obtain valid supervision, active attempts typically employ a momentum teacher for pseudo-label prediction yet observe the confirmation bias issue, where the incorrect predictions may provide wrong supervision signals and get accumulated in the training process. The primary cause of such a drawback is that the prevailing self-training framework acts as guiding the current state with previous knowledge, because the teacher is updated with the past student only. To alleviate this problem, we propose a novel self-training strategy, which allows the model to learn from the future. Concretely, at each training step, we first virtually optimize the student (i.e., caching the gradients without applying them to the model weights), then update the teacher with the virtual future student, and finally ask the teacher to produce pseudo-labels for the current student as the guidance. In this way, we manage to improve the quality of pseudo-labels and thus boost the performance. We also develop two variants of our future-self-training (FST) framework through peeping at the future both deeply (FST-D) and widely (FST-W). Taking the tasks of unsupervised domain adaptive semantic segmentation and semi-supervised semantic segmentation as the instances, we experimentally demonstrate the effectiveness and superiority of our approach under a wide range of settings. Code will be made publicly available.Comment: Accepted to NeurIPS 202

Autophagy regulates the maturation of hematopoietic precursors in the embryo

An understanding of the mechanisms regulating embryonic hematopoietic stem cell (HSC) development would facilitate their regeneration. The aorta-gonad-mesonephros region is the site for HSC production from hemogenic endothelial cells (HEC). While several distinct regulators are involved in this process, it is not yet known whether macroautophagy (autophagy) plays a role in hematopoiesis in the pre-liver stage. Here, we show that different states of autophagy exist in hematopoietic precursors and correlate with hematopoietic potential based on the LC3-RFP-EGFP mouse model. Deficiency of autophagy-related gene 5 (Atg5) specifically in endothelial cells disrupts endothelial to hematopoietic transition (EHT), by blocking the autophagic process. Using combined approaches, including single-cell RNA-sequencing (scRNA-seq), we have confirmed that Atg5 deletion interrupts developmental temporal order of EHT to further affect the pre-HSC I maturation, and that autophagy influences hemogenic potential of HEC and the formation of pre-HSC I likely via the nucleolin pathway. These findings demonstrate a role for autophagy in the formation/maturation of hematopoietic precursors.</p

Revealing unusual bandgap shifts with temperature and bandgap renormalization effect in phase-stabilized metal halide perovskites

Hybrid organic-inorganic metal halide perovskites are emerging materials in photovoltaics, whose bandgap is one of the most crucial parameters governing their light harvesting performance. Here we present temperature and photocarrier density dependence of the bandgap in two phase-stabilized perovskite thin films (MA0.3FA0.7PbI3 and MA0.3FA0.7Pb0.5Sn0.5I3) using photoluminescence and absorption spectroscopy. Contrasting bandgap shifts with temperature are observed between the two perovskites. By utilizing X-ray diffraction and in situ high pressure photoluminescence spectroscopy, we show that the thermal expansion plays only a minor role on the large bandgap blueshift due to the enhanced structural stability in our samples. Our first-principles calculations further demonstrate the significant impact of thermally induced lattice distortions on the bandgap widening and reveal that the anomalous trends are caused by the competition between the static and dynamic distortions. Additionally, both the bandgap renormalization and band filling effects are directly observed for the first time in fluence-dependent photoluminescence measurements and are employed to estimate the exciton effective mass. Our results provide new insights into the basic understanding of thermal and charge-accumulation effects on the band structure of hybrid perovskites

Autologous Skin Fibroblast-Based PLGA Nanoparticles for Treating Multiorgan Fibrosis

Fibrotic diseases remain a substantial health burden with few therapeutic approaches. A hallmark of fibrosis is the aberrant activation and accumulation of myofibroblasts, which is caused by excessive profibrotic cytokines. Conventional anticytokine therapies fail to undergo clinical trials, as simply blocking a single or several antifibrotic cytokines cannot abrogate the profibrotic microenvironment. Here, biomimetic nanoparticles based on autologous skin fibroblasts are customized as decoys to neutralize multiple fibroblast-targeted cytokines. By fusing the skin fibroblast membrane onto poly(lactic-co-glycolic) acid cores, these nanoparticles, termed fibroblast membrane-camouflaged nanoparticles (FNPs), are shown to effectively scavenge various profibrotic cytokines, including transforming growth factor-beta, interleukin (IL)-11, IL-13, and IL-17, thereby modulating the profibrotic microenvironment. FNPs are sequentially prepared into multiple formulations for different administration routines. As a proof-of-concept, in three independent animal models with various organ fibrosis (lung fibrosis, liver fibrosis, and heart fibrosis), FNPs effectively reduce the accumulation of myofibroblasts, and the formation of fibrotic tissue, concomitantly restoring organ function and indicating that FNPs are a potential broad-spectrum therapy for fibrosis management.Peer reviewe

Promoting Cardiac Repair through Simple Engineering of Nanoparticles with Exclusive Targeting Capability toward Myocardial Reperfusion Injury by Thermal Resistant Microfluidic Platform

Nanoparticle (NP)-based intravenous administration represents the most convenient cardiac targeting delivery routine, yet, there are still therapeutic issues due to the lack of targeting efficiency and specificity. Active targeting methods using functionalization of ligands onto the NPs' surface may be limited by trivial modification procedures and reduced targeting yield in vivo. Here, a microfluidics assisted single step, green synthesis method is introduced for producing targeting ligands free heart homing NPs in a tailored manner. The generated beta-glucan-based NPs exhibit precise and efficient targeting capability toward Dectin-1(+) monocytes/macrophages, which are confirmed as main pathogenesis mediators for cardiac ischemic/reperfusion (I/R) injury, with a sequentially enhanced cardiac NP accumulation, and this targeting strategy is exclusively suitable for cardiac I/R but not for other cardiovascular diseases, as confirmed both in murine and human model. Comparing to FDA-approved nano-micelles formulation, beta-glucan NPs loaded with NACHT, LRR, and PYD domains-containing protein 3 (NLRP3) inflammasome inhibitor (CY-09) exhibit better efficiency in ameliorating myocardial injury and heart failure induced by surgically induced I/R. These findings indicate a simple production of targeting-ligand free NPs, and demonstrate their potential therapeutic applications for preclinical I/R-induced cardiac injury amelioration.Peer reviewe