Multi-Mode Online Knowledge Distillation for Self-Supervised Visual Representation Learning

Self-supervised learning (SSL) has made remarkable progress in visual representation learning. Some studies combine SSL with knowledge distillation (SSL-KD) to boost the representation learning performance of small models. In this study, we propose a Multi-mode Online Knowledge Distillation method (MOKD) to boost self-supervised visual representation learning. Different from existing SSL-KD methods that transfer knowledge from a static pre-trained teacher to a student, in MOKD, two different models learn collaboratively in a self-supervised manner. Specifically, MOKD consists of two distillation modes: self-distillation and cross-distillation modes. Among them, self-distillation performs self-supervised learning for each model independently, while cross-distillation realizes knowledge interaction between different models. In cross-distillation, a cross-attention feature search strategy is proposed to enhance the semantic feature alignment between different models. As a result, the two models can absorb knowledge from each other to boost their representation learning performance. Extensive experimental results on different backbones and datasets demonstrate that two heterogeneous models can benefit from MOKD and outperform their independently trained baseline. In addition, MOKD also outperforms existing SSL-KD methods for both the student and teacher models.Comment: Accepted by CVPR 202

Semantics-Consistent Feature Search for Self-Supervised Visual Representation Learning

In contrastive self-supervised learning, the common way to learn discriminative representation is to pull different augmented "views" of the same image closer while pushing all other images further apart, which has been proven to be effective. However, it is unavoidable to construct undesirable views containing different semantic concepts during the augmentation procedure. It would damage the semantic consistency of representation to pull these augmentations closer in the feature space indiscriminately. In this study, we introduce feature-level augmentation and propose a novel semantics-consistent feature search (SCFS) method to mitigate this negative effect. The main idea of SCFS is to adaptively search semantics-consistent features to enhance the contrast between semantics-consistent regions in different augmentations. Thus, the trained model can learn to focus on meaningful object regions, improving the semantic representation ability. Extensive experiments conducted on different datasets and tasks demonstrate that SCFS effectively improves the performance of self-supervised learning and achieves state-of-the-art performance on different downstream tasks

Characterizing the supercomplex association of photosynthetic complexes in cyanobacteria

The light reactions of photosynthesis occur in thylakoid membranes that are densely packed with a series of photosynthetic complexes. The lateral organization and close association of photosynthetic complexes in native thylakoid membranes are vital for efficient light harvesting and energy transduction. Recently, analysis of the interconnections between photosynthetic complexes to form supercomplexes has garnered great interest. In this work, we report a method integrating immunoprecipitation, mass spectrometry and atomic force microscopy to identify the inter-complex associations of photosynthetic complexes in thylakoid membranes from the cyanobacterium Synechococcus elongatus PCC 7942. We characterize the preferable associations between individual photosynthetic complexes and binding proteins involved in the complex–complex interfaces, permitting us to propose the structural models of photosynthetic complex associations that promote the formation of photosynthetic supercomplexes. We also identified other potential binding proteins with the photosynthetic complexes, suggesting the highly connecting networks associated with thylakoid membranes. This study provides mechanistic insight into the physical interconnections of photosynthetic complexes and potential partners, which are crucial for efficient energy transfer and physiological acclimatization of the photosynthetic apparatus. Advanced knowledge of the protein organization and interplay of the photosynthetic machinery will inform rational design and engineering of artificial photosynthetic systems to supercharge energy production. </jats:p

Hypophyseal Involvement in Immunoglobulin G4-Related Disease: A Retrospective Study from a Single Tertiary Center

This study aims to outline the clinical features and outcomes of IgG4-related hypophysitis (IgG4-RH) patients in a tertiary medical center. We reviewed clinical manifestations and imaging and pituitary function tests at baseline, as well as during follow-up. Ten patients were included. The mean age at diagnosis of IgG4-RH was 48.4 (16.0–64.0) years. An average of 3 (0–9) extrapituitary organs were involved. Five patients had panhypopituitarism, three had only posterior hypopituitarism, one had only anterior hypopituitarism, and one had a normal pituitary function. One patient in our study had pituitary mass biopsy, lacking IgG4-positive cells despite lymphocyte infiltration forming an inflammatory pseudotumor. Five patients with a clinical course of IgG4-RH less than nine months and a whole course of IgG4-RD less than two years were managed with glucocorticoids, while three patients with a longer history were administered glucocorticoids plus immunosuppressive agents. One patient went through surgical excision, and one patient was lost to follow-up. All patients showed a prompt response clinically, but only three patients had normalized serum IgG4 levels. Two patients who took medications for less than six months relapsed. Conclusions. IgG4-RD is a broad disease, and all physicians involved have to be aware of the possibility of pituitary dysfunction. Younger patients should be expected. The histopathological feature of pituitary gland biopsy could be atypical. For patients with a longer history, the combination of GC and immunosuppressive agents is favorable. Early and adequate courses of treatment are crucial for the management of IgG4-RH. With GC and/or immunosuppressant treatment, however, pituitary function or diabetes insipidus did not improve considerably

The small subunit of DNA polymerase D (DP1) associates with GINS-GAN complex of the thermophilic archaea in Thermococcus sp. 4557.

The eukaryotic GINS, Cdc45, and minichromosome maintenance proteins form an essential complex that moves with the DNA replication fork. The GINS protein complex has also been reported to associate with DNA polymerase. In archaea, the third domain of life, DNA polymerase D (PolD) is essential for DNA replication, and the genes encoding PolDs exist only in the genomes of archaea. The archaeal GAN (GINS-associated nuclease) is believed to be a homolog of the eukaryotic Cdc45. In this study, we found that the Thermococcus sp. 4557 DP1 (small subunit of PolD) interacted with GINS15 in vitro, and the 3’-5’ exonuclease activity of DP1 was inhibited by GINS15. We also demonstrated that the GAN, GINS15, and DP1 proteins interact to form a complex adapting a GAN-GINS15-DP1 order. The results of this study imply that the complex constitutes a core of the DNA replisome in archaea

Phase Transformation Driven by Oxygen Vacancy Redistribution as the Mechanism of Ferroelectric Hf0.5Zr0.5O2 Fatigue

AbstractAs a promising candidate for nonvolatile memory devices, the hafnia‐based ferroelectric system has recently been a hot research topic. Although significant progress has been made over the past decade, the endurance problem is still an obstacle to its final application. In perovskite‐based ferroelectrics, such as the well‐studied Pb[ZrxTi1−x]O3 (PZT) family, polarization fatigue has been discussed within the framework of the interaction of charged defects (such as oxygen vacancies) with the moving domains during the switching process, particularly at the electrode‐ferroelectric interface. Armed with this background, a hypothesis is set out to test that a similar mechanism can be in play with the hafnia‐based ferroelectrics. The conducting perovskite La‐Sr‐Mn‐O is used as the contact electrode to create La0.67Sr0.33MnO3 / Hf0.5Zr0.5O2 (HZO)/ La0.67Sr0.33MnO3 capacitor structures deposited on SrTiO3‐Si substrates. Nanoscale X‐ray diffraction is performed on single capacitors, and a structural phase transition from polar o‐phase toward non‐polar m‐phase is demonstrated during the bipolar switching process. The energy landscape of multiphase HZO has been calculated at varying oxygen vacancy concentrations. Based on both theoretical and experimental results, it is found that a polar to non‐polar phase transformation caused by oxygen vacancy redistribution during electric cycling is a likely explanation for fatigue in HZO