Sample-efficient Reinforcement Learning Representation Learning with Curiosity Contrastive Forward Dynamics Model

Developing an agent in reinforcement learning (RL) that is capable of performing complex control tasks directly from high-dimensional observation such as raw pixels is yet a challenge as efforts are made towards improving sample efficiency and generalization. This paper considers a learning framework for Curiosity Contrastive Forward Dynamics Model (CCFDM) in achieving a more sample-efficient RL based directly on raw pixels. CCFDM incorporates a forward dynamics model (FDM) and performs contrastive learning to train its deep convolutional neural network-based image encoder (IE) to extract conducive spatial and temporal information for achieving a more sample efficiency for RL. In addition, during training, CCFDM provides intrinsic rewards, produced based on FDM prediction error, encourages the curiosity of the RL agent to improve exploration. The diverge and less-repetitive observations provide by both our exploration strategy and data augmentation available in contrastive learning improve not only the sample efficiency but also the generalization. Performance of existing model-free RL methods such as Soft Actor-Critic built on top of CCFDM outperforms prior state-of-the-art pixel-based RL methods on the DeepMind Control Suite benchmark

SoftGroup++: Scalable 3D Instance Segmentation with Octree Pyramid Grouping

Existing state-of-the-art 3D point cloud instance segmentation methods rely on a grouping-based approach that groups points to obtain object instances. Despite improvement in producing accurate segmentation results, these methods lack scalability and commonly require dividing large input into multiple parts. To process a scene with millions of points, the existing fastest method SoftGroup \cite{vu2022softgroup} requires tens of seconds, which is under satisfaction. Our finding is that $k$-Nearest Neighbor ($k$-NN), which serves as the prerequisite of grouping, is a computational bottleneck. This bottleneck severely worsens the inference time in the scene with a large number of points. This paper proposes SoftGroup++ to address this computational bottleneck and further optimize the inference speed of the whole network. SoftGroup++ is built upon SoftGroup, which differs in three important aspects: (1) performs octree $k$-NN instead of vanilla $k$-NN to reduce time complexity from $\mathcal{O}(n^2)$ to $\mathcal{O}(n \log n)$, (2) performs pyramid scaling that adaptively downsamples backbone outputs to reduce search space for $k$-NN and grouping, and (3) performs late devoxelization that delays the conversion from voxels to points towards the end of the model such that intermediate components operate at a low computational cost. Extensive experiments on various indoor and outdoor datasets demonstrate the efficacy of the proposed SoftGroup++. Notably, SoftGroup++ processes large scenes of millions of points by a single forward without dividing the input into multiple parts, thus enriching contextual information. Especially, SoftGroup++ achieves 2.4 points AP$_{50}$ improvement while nearly $6\times$ faster than the existing fastest method on S3DIS dataset. The code and trained models will be made publicly available.Comment: Technical repor

Regulation of SOX11 expression through CCND1 and STAT3 in mantle cell lymphoma

The neural transcription factor SOX11 is usually highly expressed in typical mantle cell lymphoma (MCL), but it is absent in the more indolent form of MCL. Despite being an important diagnostic marker for this hard-to-treat malignancy, the mechanisms of aberrant SOX11 expression are largely unknown. Herein, we describe 2 modes of SOX11 regulation by the cell-cycle regulator cyclin D1 (CCND1) and the signal transducer and activator of transcription 3 (STAT3). We found that ectopic expression of CCND1 in multiple human MCL cell lines resulted in increased SOX11 transcription, which correlated with increased acetylated histones H3K9 and H3K14 (H3K9/14Ac). Increased H3K9/14Ac and SOX11 expression was also observed after histone deacetylase 1 (HDAC1) or HDAC2 was depleted by RNA interference or inhibited by the HDAC inhibitor vorinostat. Mechanistically, we showed that CCND1 interacted with and sequestered HDAC1 and HDAC2 from the SOX11 locus, leading to SOX11 upregulation. Interestingly, our data revealed a potential inverse relationship between phosphorylated Y705 STAT3 and SOX11 expression in MCL cell lines, primary tumors, and patient-derived xenografts. Functionally, inactivation of STAT3 by inhibiting the upstream Janus kinase (JAK) 1 or JAK2 or by STAT3 knockdown was found to increase SOX11 expression, whereas interleukin-21 (IL-21)–induced STAT3 activation or overexpression of the constitutively active form of STAT3 decreased SOX11 expression. In addition, targeting SOX11 directly by RNA interference or indirectly by IL-21 treatment induced toxicity in SOX11^+ MCL cells. Collectively, we demonstrate the involvement of CCND1 and STAT3 in the regulation of SOX11 expression, providing new insights and therapeutic implications in MCL

Regulation of SOX11 expression through CCND1 and STAT3 in mantle cell lymphoma

The neural transcription factor SOX11 is usually highly expressed in typical mantle cell lymphoma (MCL), but it is absent in the more indolent form of MCL. Despite being an important diagnostic marker for this hard-to-treat malignancy, the mechanisms of aberrant SOX11 expression are largely unknown. Herein, we describe 2 modes of SOX11 regulation by the cell-cycle regulator cyclin D1 (CCND1) and the signal transducer and activator of transcription 3 (STAT3). We found that ectopic expression of CCND1 in multiple human MCL cell lines resulted in increased SOX11 transcription, which correlated with increased acetylated histones H3K9 and H3K14 (H3K9/14Ac). Increased H3K9/14Ac and SOX11 expression was also observed after histone deacetylase 1 (HDAC1) or HDAC2 was depleted by RNA interference or inhibited by the HDAC inhibitor vorinostat. Mechanistically, we showed that CCND1 interacted with and sequestered HDAC1 and HDAC2 from the SOX11 locus, leading to SOX11 upregulation. Interestingly, our data revealed a potential inverse relationship between phosphorylated Y705 STAT3 and SOX11 expression in MCL cell lines, primary tumors, and patient-derived xenografts. Functionally, inactivation of STAT3 by inhibiting the upstream Janus kinase (JAK) 1 or JAK2 or by STAT3 knockdown was found to increase SOX11 expression, whereas interleukin-21 (IL-21)–induced STAT3 activation or overexpression of the constitutively active form of STAT3 decreased SOX11 expression. In addition, targeting SOX11 directly by RNA interference or indirectly by IL-21 treatment induced toxicity in SOX11^+ MCL cells. Collectively, we demonstrate the involvement of CCND1 and STAT3 in the regulation of SOX11 expression, providing new insights and therapeutic implications in MCL

Clinical and Virological Study of Dengue Cases and the Members of Their Households: The Multinational DENFRAME Project

Dengue is the most important mosquito-borne viral disease in humans. This disease is now endemic in more than 100 countries and threatens more than 2.5 billion people living in tropical countries. It currently affects about 50 to 100 million people each year. It causes a wide range of symptoms, from an inapparent to mild dengue fever, to severe forms, including dengue hemorrhagic fever. Currently no specific vaccine or antiviral drugs are available. We carried out a prospective clinical study in South-East Asia and Latin America, of virologically confirmed dengue-infected patients attending the hospital, and members of their households. Among 215 febrile dengue subjects, 177 agreed to household investigation. Based on our data, we estimated the proportion of dengue-infected household members to be about 45%. At the time of the home visit, almost three quarters of (29/39) presented an inapparent dengue infection. The proportion of inapparent dengue infection was higher in South-East Asia than in Latin America. These findings confirm the complexity of dengue disease in humans and the need to strengthen multidisciplinary research efforts to improve our understanding of virus transmission and host responses to dengue virus in various human populations

Learning Imbalanced Datasets with Maximum Margin Loss

A learning algorithm referred to as Maximum Margin (MM) is proposed for considering the class-imbalance data learning issue: the trained model tends to predict the majority of classes rather than the minority ones. That is, underfitting for minority classes seems to be one of the challenges of generalization. For a good generalization of the minority classes, we design a new Maximum Margin (MM) loss function, motivated by minimizing a margin-based generalization bound through the shifting decision bound. The theoretically-principled label-distribution-aware margin (LDAM) loss was successfully applied with prior strategies such as re-weighting or re-sampling along with the effective training schedule. However, they did not investigate the maximum margin loss function yet. In this study, we investigate the performances of two types of hard maximum margin-based decision boundary shift with LDAM's training schedule on artificially imbalanced CIFAR-10/100 for fair comparisons and effectiveness

SCNet: Training Inference Sample Consistency for Instance Segmentation

Cascaded architectures have brought significant performance improvement in object detection and instance segmentation. However, there are lingering issues regarding the disparity in the Intersection-over-Union (IoU) distribution of the samples between training and inference. This disparity can potentially exacerbate detection accuracy. This paper proposes an architecture referred to as Sample Consistency Network (SCNet) to ensure that the IoU distribution of the samples at training time is close to that at inference time. Furthermore, SCNet incorporates feature relay and utilizes global contextual information to further reinforce the reciprocal relationships among classifying, detecting, and segmenting sub-tasks. Extensive experiments on the standard COCO dataset reveal the effectiveness of the proposed method over multiple evaluation metrics, including box AP, mask AP, and inference speed. In particular, while running 38\% faster, the proposed SCNet improves the AP of the box and mask predictions by respectively 1.3 and 2.3 points compared to the strong Cascade Mask R-CNN baseline. Code is available at https://github.com/thangvubk/SCNet