SSC-RS: Elevate LiDAR Semantic Scene Completion with Representation Separation and BEV Fusion

Semantic scene completion (SSC) jointly predicts the semantics and geometry of the entire 3D scene, which plays an essential role in 3D scene understanding for autonomous driving systems. SSC has achieved rapid progress with the help of semantic context in segmentation. However, how to effectively exploit the relationships between the semantic context in semantic segmentation and geometric structure in scene completion remains under exploration. In this paper, we propose to solve outdoor SSC from the perspective of representation separation and BEV fusion. Specifically, we present the network, named SSC-RS, which uses separate branches with deep supervision to explicitly disentangle the learning procedure of the semantic and geometric representations. And a BEV fusion network equipped with the proposed Adaptive Representation Fusion (ARF) module is presented to aggregate the multi-scale features effectively and efficiently. Due to the low computational burden and powerful representation ability, our model has good generality while running in real-time. Extensive experiments on SemanticKITTI demonstrate our SSC-RS achieves state-of-the-art performance.Comment: 8 pages, 5 figures, IROS202

CR-SFP: Learning Consistent Representation for Soft Filter Pruning

Soft filter pruning~(SFP) has emerged as an effective pruning technique for allowing pruned filters to update and the opportunity for them to regrow to the network. However, this pruning strategy applies training and pruning in an alternative manner, which inevitably causes inconsistent representations between the reconstructed network~(R-NN) at the training and the pruned network~(P-NN) at the inference, resulting in performance degradation. In this paper, we propose to mitigate this gap by learning consistent representation for soft filter pruning, dubbed as CR-SFP. Specifically, for each training step, CR-SFP optimizes the R-NN and P-NN simultaneously with different distorted versions of the same training data, while forcing them to be consistent by minimizing their posterior distribution via the bidirectional KL-divergence loss. Meanwhile, the R-NN and P-NN share backbone parameters thus only additional classifier parameters are introduced. After training, we can export the P-NN for inference. CR-SFP is a simple yet effective training framework to improve the accuracy of P-NN without introducing any additional inference cost. It can also be combined with a variety of pruning criteria and loss functions. Extensive experiments demonstrate our CR-SFP achieves consistent improvements across various CNN architectures. Notably, on ImageNet, our CR-SFP reduces more than 41.8\% FLOPs on ResNet18 with 69.2\% top-1 accuracy, improving SFP by 2.1\% under the same training settings. The code will be publicly available on GitHub.Comment: 11 pages, 4 figure

PANet: LiDAR Panoptic Segmentation with Sparse Instance Proposal and Aggregation

Reliable LiDAR panoptic segmentation (LPS), including both semantic and instance segmentation, is vital for many robotic applications, such as autonomous driving. This work proposes a new LPS framework named PANet to eliminate the dependency on the offset branch and improve the performance on large objects, which are always over-segmented by clustering algorithms. Firstly, we propose a non-learning Sparse Instance Proposal (SIP) module with the ``sampling-shifting-grouping" scheme to directly group thing points into instances from the raw point cloud efficiently. More specifically, balanced point sampling is introduced to generate sparse seed points with more uniform point distribution over the distance range. And a shift module, termed bubble shifting, is proposed to shrink the seed points to the clustered centers. Then we utilize the connected component label algorithm to generate instance proposals. Furthermore, an instance aggregation module is devised to integrate potentially fragmented instances, improving the performance of the SIP module on large objects. Extensive experiments show that PANet achieves state-of-the-art performance among published works on the SemanticKITII validation and nuScenes validation for the panoptic segmentation task.Comment: 8 pages, 3 figures, IROS202

A Randomized Controlled Trial of Conbercept Pretreatment before Vitrectomy in Proliferative Diabetic Retinopathy

Purpose. To determine the efficacy and safety of preoperative intravitreal conbercept (IVC) injection before vitrectomy for proliferative diabetic retinopathy (PDR). Methods. 107 eyes of 88 patients that underwent pars plana vitrectomy (PPV) for active PDR were enrolled. All patients were assigned randomly to either preoperative IVC group or control group. Follow-up examinations were performed for three months after surgery. The primary bioactivity measures were severity of intraoperative bleeding, incidence of early and late recurrent VH, vitreous clear-up time, and best-corrected visual acuity (BCVA) levels. The secondary safety measures included intraocular pressure, endophthalmitis, rubeosis, tractional retinal detachment, and systemic adverse events. Results. The incidence and severity of intraoperative bleeding were significantly lower in IVC group than in the control group. The average vitreous clear-up time of early recurrent VH was significantly shorter in IVC group compared with that in control group. There was no significant difference in vitreous clear-up time of late recurrent VH between the two groups. Patients that received pretreatment of conbercept had much better BCVA at 3 days, 1 week, and 1 month after surgery than control group. Moreover, both patients with improved BCVA were greater in IVC group than in control group at each follow-up. Conclusions. Conbercept pretreatment could be an effective adjunct to vitrectomy in accelerating postoperative vitreous clear-up and acquiring stable visual acuity restoration for PDR

A Multimodal, Multi-Task Adapting Framework for Video Action Recognition

Recently, the rise of large-scale vision-language pretrained models like CLIP, coupled with the technology of Parameter-Efficient FineTuning (PEFT), has captured substantial attraction in video action recognition. Nevertheless, prevailing approaches tend to prioritize strong supervised performance at the expense of compromising the models' generalization capabilities during transfer. In this paper, we introduce a novel Multimodal, Multi-task CLIP adapting framework named M2-CLIP to address these challenges, preserving both high supervised performance and robust transferability. Firstly, to enhance the individual modality architectures, we introduce multimodal adapters to both the visual and text branches. Specifically, we design a novel visual TED-Adapter, that performs global Temporal Enhancement and local temporal Difference modeling to improve the temporal representation capabilities of the visual encoder. Moreover, we adopt text encoder adapters to strengthen the learning of semantic label information. Secondly, we design a multi-task decoder with a rich set of supervisory signals, including the original contrastive learning head, a cross-modal classification head, a cross-modal masked language modeling head, and a visual classification head. This multi-task decoder adeptly satisfies the need for strong supervised performance within a multimodal framework. Experimental results validate the efficacy of our approach, demonstrating exceptional performance in supervised learning while maintaining strong generalization in zero-shot scenarios

PRL3-zumab as an immunotherapy to inhibit tumors expressing PRL3 oncoprotein

Tumor-specific antibody drugs can serve as cancer therapy with minimal side effects. A humanized antibody, PRL3-zumab, specifically binds to an intracellular oncogenic phosphatase PRL3, which is frequently expressed in several cancers. Here we show that PRL3-zumab specifically inhibits PRL3+ cancer cells in vivo, but not in vitro. PRL3 antigens are detected on the cell surface and outer exosomal membranes, implying an ‘inside-out’ externalization of PRL3. PRL3-zumab binds to surface PRL3 in a manner consistent with that in classical antibody-dependent cell-mediated cytotoxicity or antibody-dependent cellular phagocytosis tumor elimination pathways, as PRL3-zumab requires an intact Fc region and host FcγII/III receptor engagement to recruit B cells, NK cells and macrophages to PRL3+ tumor microenvironments. PRL3 is overexpressed in 80.6% of 151 fresh-frozen tumor samples across 11 common cancers examined, but not in patient-matched normal tissues, thereby implicating PRL3 as a tumor-associated antigen. Targeting externalized PRL3 antigens with PRL3-zumab may represent a feasible approach for anti-tumor immunotherapy.ASTAR (Agency for Sci., Tech. and Research, S’pore)Published versio

PRL3-zumab as an immunotherapy to inhibit tumors expressing PRL3 oncoprotein (vol 10, 2484, 2019)

10.1038/s41467-021-26548-6NATURE COMMUNICATIONS12