Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition

The teacher-free online Knowledge Distillation (KD) aims to train an ensemble of multiple student models collaboratively and distill knowledge from each other. Although existing online KD methods achieve desirable performance, they often focus on class probabilities as the core knowledge type, ignoring the valuable feature representational information. We present a Mutual Contrastive Learning (MCL) framework for online KD. The core idea of MCL is to perform mutual interaction and transfer of contrastive distributions among a cohort of networks in an online manner. Our MCL can aggregate cross-network embedding information and maximize the lower bound to the mutual information between two networks. This enables each network to learn extra contrastive knowledge from others, leading to better feature representations, thus improving the performance of visual recognition tasks. Beyond the final layer, we extend MCL to intermediate layers and perform an adaptive layer-matching mechanism trained by meta-optimization. Experiments on image classification and transfer learning to visual recognition tasks show that layer-wise MCL can lead to consistent performance gains against state-of-the-art online KD approaches. The superiority demonstrates that layer-wise MCL can guide the network to generate better feature representations. Our code is publicly avaliable at https://github.com/winycg/L-MCL.Comment: 18 pages, accepted by IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI-2023

Lung Nodule Segmentation and Uncertain Region Prediction with an Uncertainty-Aware Attention Mechanism

Radiologists possess diverse training and clinical experiences, leading to variations in the segmentation annotations of lung nodules and resulting in segmentation uncertainty.Conventional methods typically select a single annotation as the learning target or attempt to learn a latent space comprising multiple annotations. However, these approaches fail to leverage the valuable information inherent in the consensus and disagreements among the multiple annotations. In this paper, we propose an Uncertainty-Aware Attention Mechanism (UAAM) that utilizes consensus and disagreements among multiple annotations to facilitate better segmentation. To this end, we introduce the Multi-Confidence Mask (MCM), which combines a Low-Confidence (LC) Mask and a High-Confidence (HC) Mask.The LC mask indicates regions with low segmentation confidence, where radiologists may have different segmentation choices. Following UAAM, we further design an Uncertainty-Guide Multi-Confidence Segmentation Network (UGMCS-Net), which contains three modules: a Feature Extracting Module that captures a general feature of a lung nodule, an Uncertainty-Aware Module that produces three features for the the annotations' union, intersection, and annotation set, and an Intersection-Union Constraining Module that uses distances between the three features to balance the predictions of final segmentation and MCM. To comprehensively demonstrate the performance of our method, we propose a Complex Nodule Validation on LIDC-IDRI, which tests UGMCS-Net's segmentation performance on lung nodules that are difficult to segment using common methods. Experimental results demonstrate that our method can significantly improve the segmentation performance on nodules that are difficult to segment using conventional methods.Comment: 10 pages, 10 figures. We have reported a preliminary version of this work in MICCAI 202

Expression of phospho-ERK1/2 and PI3-K in benign and malignant gallbladder lesions and its clinical and pathological correlations

Abstract Background An increasing number of studies have shown that ERK and PI3-K/AKT signaling pathways are involved in various human cancers including hepatocellular carcinoma and cholangiocarcinoma. However, few studies have examined gallbladder cancer specimens, and little is known about the clinical and pathological significance of ERK1/2 and PI3-K/AKT signaling changes in gallbladder adenocarcinoma. In this study, we examined phospho-ERK1/2 (p-ERK1/2) and PI3K expression and analyzed its clinicopathological impact in gallbladder adenocarcinoma. Methods Immunohistochemistry was used to detect and compare the frequency of p-ERK1/2 and PI3-K expression in gallbladder adenocarcinoma, peri-tumor tissues, adenomatous polyps, and chronic cholecystitis specimens. Results The positive staining for p-EKR1/2 and PI3-K were 63/108 (58.3%) and 55/108 (50.9%) in gallbladder adenocarcinoma; 14/46 (30.4%) and 5/46 (10.1%) in peri-tumor tissues; 3/15 (20%) and 3/15 (20%) in adenomatous polyps; and 4/35 (11.4%) and 3/35 (8.6%) in chronic cholecystitis. The positive rate of p-ERK1/2 or PI3-K in gallbladder adenocarcinoma was significantly higher than that in peri-tumor tissue (both, P P P P P P P P = 0.062) was associated with decreased overall survival. Multivariate Cox regression analysis showed that increased p-ERK1/2 expression was an independent prognostic predictor in gallbladder carcinoma (P = 0.028). Conclusion Increased expression of p-ERK1/2 and PI3K might contribute to gallbladder carcinogenesis. p-ERK1/2 over-expression is correlated with decreased survival and therefore may serve as an important biological marker in development of gallbladder adenocarcinoma.</p

Bacterial Lifestyle in a Deep-sea Hydrothermal Vent Chimney Revealed by the Genome Sequence of the Thermophilic Bacterium Deferribacter desulfuricans SSM1

The complete genome sequence of the thermophilic sulphur-reducing bacterium, Deferribacter desulfuricans SMM1, isolated from a hydrothermal vent chimney has been determined. The genome comprises a single circular chromosome of 2 234 389 bp and a megaplasmid of 308 544 bp. Many genes encoded in the genome are most similar to the genes of sulphur- or sulphate-reducing bacterial species within Deltaproteobacteria. The reconstructed central metabolisms showed a heterotrophic lifestyle primarily driven by C1 to C3 organics, e.g. formate, acetate, and pyruvate, and also suggested that the inability of autotrophy via a reductive tricarboxylic acid cycle may be due to the lack of ATP-dependent citrate lyase. In addition, the genome encodes numerous genes for chemoreceptors, chemotaxis-like systems, and signal transduction machineries. These signalling networks may be linked to this bacterium's versatile energy metabolisms and may provide ecophysiological advantages for D. desulfuricans SSM1 thriving in the physically and chemically fluctuating environments near hydrothermal vents. This is the first genome sequence from the phylum Deferribacteres

Evolutionary history of the OmpR/IIIA family of signal transduction two component systems in Lactobacillaceae and Leuconostocaceae

<p>Abstract</p> <p>Background</p> <p>Two component systems (TCS) are signal transduction pathways which typically consist of a sensor histidine kinase (HK) and a response regulator (RR). In this study, we have analyzed the evolution of TCS of the OmpR/IIIA family in <it>Lactobacillaceae </it>and <it>Leuconostocaceae</it>, two families belonging to the group of lactic acid bacteria (LAB). LAB colonize nutrient-rich environments such as foodstuffs, plant materials and the gastrointestinal tract of animals thus driving the study of this group of both basic and applied interest.</p> <p>Results</p> <p>The genomes of 19 strains belonging to 16 different species have been analyzed. The number of TCS encoded by the strains considered in this study varied between 4 in <it>Lactobacillus helveticus </it>and 17 in <it>Lactobacillus casei</it>. The OmpR/IIIA family was the most prevalent in <it>Lactobacillaceae </it>accounting for 71% of the TCS present in this group. The phylogenetic analysis shows that no new TCS of this family has recently evolved in these <it>Lactobacillaceae </it>by either lineage-specific gene expansion or domain shuffling. Furthermore, no clear evidence of non-orthologous replacements of either RR or HK partners has been obtained, thus indicating that coevolution of cognate RR and HKs has been prevalent in <it>Lactobacillaceae</it>.</p> <p>Conclusions</p> <p>The results obtained suggest that vertical inheritance of TCS present in the last common ancestor and lineage-specific gene losses appear as the main evolutionary forces involved in their evolution in <it>Lactobacillaceae</it>, although some HGT events cannot be ruled out. This would agree with the genomic analyses of <it>Lactobacillales </it>which show that gene losses have been a major trend in the evolution of this group.</p