OVSNet : Towards One-Pass Real-Time Video Object Segmentation

Video object segmentation aims at accurately segmenting the target object regions across consecutive frames. It is technically challenging for coping with complicated factors (e.g., shape deformations, occlusion and out of the lens). Recent approaches have largely solved them by using backforth re-identification and bi-directional mask propagation. However, their methods are extremely slow and only support offline inference, which in principle cannot be applied in real time. Motivated by this observation, we propose a efficient detection-based paradigm for video object segmentation. We propose an unified One-Pass Video Segmentation framework (OVS-Net) for modeling spatial-temporal representation in a unified pipeline, which seamlessly integrates object detection, object segmentation, and object re-identification. The proposed framework lends itself to one-pass inference that effectively and efficiently performs video object segmentation. Moreover, we propose a maskguided attention module for modeling the multi-scale object boundary and multi-level feature fusion. Experiments on the challenging DAVIS 2017 demonstrate the effectiveness of the proposed framework with comparable performance to the state-of-the-art, and the great efficiency about 11.5 FPS towards pioneering real-time work to our knowledge, more than 5 times faster than other state-of-the-art methods.Comment: 10 pages, 6 figure

Local-to-Global Information Communication for Real-Time Semantic Segmentation Network Search

Neural Architecture Search (NAS) has shown great potentials in automatically designing neural network architectures for real-time semantic segmentation. Unlike previous works that utilize a simplified search space with cell-sharing way, we introduce a new search space where a lightweight model can be more effectively searched by replacing the cell-sharing manner with cell-independent one. Based on this, the communication of local to global information is achieved through two well-designed modules. For local information exchange, a graph convolutional network (GCN) guided module is seamlessly integrated as a communication deliver between cells. For global information aggregation, we propose a novel dense-connected fusion module (cell) which aggregates long-range multi-level features in the network automatically. In addition, a latency-oriented constraint is endowed into the search process to balance the accuracy and latency. We name the proposed framework as Local-to-Global Information Communication Network Search (LGCNet). Extensive experiments on Cityscapes and CamVid datasets demonstrate that LGCNet achieves the new state-of-the-art trade-off between accuracy and speed. In particular, on Cityscapes dataset, LGCNet achieves the new best performance of 74.0\% mIoU with the speed of 115.2 FPS on Titan Xp.Comment: arXiv admin note: text overlap with arXiv:1909.0679

Graph-guided Architecture Search for Real-time Semantic Segmentation

Designing a lightweight semantic segmentation network often requires researchers to find a trade-off between performance and speed, which is always empirical due to the limited interpretability of neural networks. In order to release researchers from these tedious mechanical trials, we propose a Graph-guided Architecture Search (GAS) pipeline to automatically search real-time semantic segmentation networks. Unlike previous works that use a simplified search space and stack a repeatable cell to form a network, we introduce a novel search mechanism with new search space where a lightweight model can be effectively explored through the cell-level diversity and latencyoriented constraint. Specifically, to produce the cell-level diversity, the cell-sharing constraint is eliminated through the cell-independent manner. Then a graph convolution network (GCN) is seamlessly integrated as a communication mechanism between cells. Finally, a latency-oriented constraint is endowed into the search process to balance the speed and performance. Extensive experiments on Cityscapes and CamVid datasets demonstrate that GAS achieves the new state-of-the-art trade-off between accuracy and speed. In particular, on Cityscapes dataset, GAS achieves the new best performance of 73.5% mIoU with speed of 108.4 FPS on Titan Xp.Comment: CVPR202

Macrophage PPARg inhibits Gpr132 to mediate the anti-tumor effects of rosiglitazone

Abstract Tumor-associated macrophage (TAM) significantly contributes to cancer progression. Human cancer is enhanced by PPARg loss-of-function mutations, but inhibited by PPARg agonists such as TZD diabetes drugs including rosiglitazone. However, it remains enigmatic whether and how macrophage contributes to PPARg tumor-suppressive functions. Here we report that macrophage PPARg deletion in mice not only exacerbates mammary tumor development but also impairs the anti-tumor effects of rosiglitazone. Mechanistically, we identify Gpr132 as a novel direct PPARg target in macrophage whose expression is enhanced by PPARg loss but repressed by PPARg activation. Functionally, macrophage Gpr132 is pro-inflammatory and pro-tumor. Genetic Gpr132 deletion not only retards inflammation and cancer growth but also abrogates the anti-tumor effects of PPARg and rosiglitazone. Pharmacological Gpr132 inhibition significantly impedes mammary tumor malignancy. These findings uncover macrophage PPARg and Gpr132 as critical TAM modulators, new cancer therapeutic targets, and essential mediators of TZD anti-cancer effects

Paired protein kinases PRKCI-RIPK2 promote pancreatic cancer growth and metastasis via enhancing NF-κB/JNK/ERK phosphorylation

Abstract Background Protein kinases play a pivotal role in the malignant evolution of pancreatic cancer (PC) through mediating phosphorylation. Many kinase inhibitors have been developed and translated into clinical use, while the complex pathology of PC confounds their clinical efficacy and warrants the discovery of more effective therapeutic targets. Methods Here, we used the Gene Expression Omnibus (GEO) database and protein kinase datasets to map the PC-related protein kinase-encoding genes. Then, applying Gene Expression and Profiling Interactive Analysis (GEPIA), GEO and Human Protein Atlas, we evaluated gene correlation, gene expression at protein and mRNA levels, as well as survival significance. In addition, we performed protein kinase RIPK2 knockout and overexpression to observe effects of its expression on PC cell proliferation, migration and invasion in vitro, as well as cell apoptosis, reactive oxygen species (ROS) production and autophagy. We established PC subcutaneous xenograft and liver metastasis models to investigate the effects of RIPK2 knockout on PC growth and metastasis. Co-immunoprecipitation and immunofluorescence were utilized to explore the interaction between protein kinases RIPK2 and PRKCI. Polymerase chain reaction and immunoblotting were used to evaluate gene expression and protein phosphorylation level. Results We found fourteen kinases aberrantly expressed in human PC and nine kinases with prognosis significance. Among them, RIPK2 with both serine/threonine and tyrosine activities were validated to promote PC cells proliferation, migration and invasion. RIPK2 knockout could inhibit subcutaneous tumor growth and liver metastasis of PC. In addition, RIPK2 knockout suppressed autophagosome formation, increased ROS production and PC cell apoptosis. Importantly, another oncogenic kinase PRKCI could interact with RIPK2 to enhance the phosphorylation of downstream NF-κB, JNK and ERK. Conclusion Paired protein kinases PRKCI-RIPK2 with multiple phosphorylation activities represent a new pathological mechanism in PC and could provide potential targets for PC therapy

Smoking‐Induced M2‐TAMs, via circEML4 in EVs, Promote the Progression of NSCLC through ALKBH5‐Regulated m6A Modification of SOCS2 in NSCLC Cells

Abstract Lung cancer is a commonly diagnosed disease worldwide, with non‐small cell lung cancers (NSCLCs) accounting for ≈ 85% of cases. Cigarette smoke is an environmental exposure promoting progression of NSCLC, but its role is poorly understood. This study reports that smoking‐induced accumulation of M2‐type tumor‐associated macrophages (M2‐TAMs) surrounding NSCLC tissues promotes malignancy. Specifically, extracellular vesicles (EVs) from cigarette smoke extract (CSE)‐induced M2 macrophages promoted malignancy of NSCLC cells in vitro and in vivo. circEML4 in EVs from CSE‐induced M2 macrophages is transported to NSCLC cells, where it reduced the distribution of ALKBH5 in the nucleus by interacting with Human AlkB homolog H5 (ALKBH5), resulting in elevated N6‐methyladenosine (m6A) modifications. m6A‐seq and RNA‐seq revealed suppressor of cytokine signaling 2 (SOCS2)‐mediated activation of the Janus kinase‐signal transducer and activator of transcription (JAK‐STAT) pathway by regulating m6A modification of SOCS2 via ALKBH5. Down‐regulation of circEML4 in EVs from CSE‐induced M2 macrophages reversed EVs‐enhanced tumorigenicity and metastasis in NSCLC cells. Furthermore, this study found that smoking patients showed an increase in circEML4‐positive M2‐TAMs. These results indicate that smoking‐induced M2‐TAMs via circEML4 in EVs promote the NSCLC progression through ALKBH5‐regulated m6A modification of SOCS2. This study also reveals that circEML4 in EVs from TAMs acts as a diagnostic biomarker for NSCLC, especially for patients with smoking history