Contrastive Learning Is Spectral Clustering On Similarity Graph

Contrastive learning is a powerful self-supervised learning method, but we have a limited theoretical understanding of how it works and why it works. In this paper, we prove that contrastive learning with the standard InfoNCE loss is equivalent to spectral clustering on the similarity graph. Using this equivalence as the building block, we extend our analysis to the CLIP model and rigorously characterize how similar multi-modal objects are embedded together. Motivated by our theoretical insights, we introduce the kernel mixture loss, incorporating novel kernel functions that outperform the standard Gaussian kernel on several vision datasets.Comment: We express our gratitude to the anonymous reviewers for their valuable feedbac

RelationMatch: Matching In-batch Relationships for Semi-supervised Learning

Semi-supervised learning has achieved notable success by leveraging very few labeled data and exploiting the wealth of information derived from unlabeled data. However, existing algorithms usually focus on aligning predictions on paired data points augmented from an identical source, and overlook the inter-point relationships within each batch. This paper introduces a novel method, RelationMatch, which exploits in-batch relationships with a matrix cross-entropy (MCE) loss function. Through the application of MCE, our proposed method consistently surpasses the performance of established state-of-the-art methods, such as FixMatch and FlexMatch, across a variety of vision datasets. Notably, we observed a substantial enhancement of 15.21% in accuracy over FlexMatch on the STL-10 dataset using only 40 labels. Moreover, we apply MCE to supervised learning scenarios, and observe consistent improvements as well

Kernel-SSL: Kernel KL Divergence for Self-Supervised Learning

Contrastive learning usually compares one positive anchor sample with lots of negative samples to perform Self-Supervised Learning (SSL). Alternatively, non-contrastive learning, as exemplified by methods like BYOL, SimSiam, and Barlow Twins, accomplishes SSL without the explicit use of negative samples. Inspired by the existing analysis for contrastive learning, we provide a reproducing kernel Hilbert space (RKHS) understanding of many existing non-contrastive learning methods. Subsequently, we propose a novel loss function, Kernel-SSL, which directly optimizes the mean embedding and the covariance operator within the RKHS. In experiments, our method Kernel-SSL outperforms state-of-the-art methods by a large margin on ImageNet datasets under the linear evaluation settings. Specifically, when performing 100 epochs pre-training, our method outperforms SimCLR by 4.6%

Information Flow in Self-Supervised Learning

In this paper, we provide a comprehensive toolbox for understanding and enhancing self-supervised learning (SSL) methods through the lens of matrix information theory. Specifically, by leveraging the principles of matrix mutual information and joint entropy, we offer a unified analysis for both contrastive and feature decorrelation based methods. Furthermore, we propose the matrix variational masked auto-encoder (M-MAE) method, grounded in matrix information theory, as an enhancement to masked image modeling. The empirical evaluations underscore the effectiveness of M-MAE compared with the state-of-the-art methods, including a 3.9% improvement in linear probing ViT-Base, and a 1% improvement in fine-tuning ViT-Large, both on ImageNet

The impact of TSC-1 and -2 mutations on response to therapy in malignant PEComa: A multicenter retrospective analysis

BACKGROUND: Perivascular epithelioid cell neoplasms (PEComas) are a diverse family of mesenchymal tumors with myomelanocytic differentiation that disproportionately affect women and can be associated with tuberous sclerosis (TS). Although mTOR inhibition is widely used as first-line treatment, it is unclear what genomic alterations exist in these tumors and how they influence the response to therapy. METHODS: This was a multicenter study conducted at five sites within the US. The data were collected from 1 January 2004 to 31 January 2021. We conducted a retrospective analysis to identify PEComa patients with next-generation sequencing (NGS) data and compared outcomes based on mutations. RESULTS: No significant differences in survival were identified between CONCLUSIONS: We were unable to detect differences in survival based on genomic alterations or PFS between mTOR inhibition versus other systemic therapies. Future studies should seek to identify other drivers o

Targeted Cyclo[8]pyrrole-Based NIR-II Photoacoustic Tomography Probe for Suppression of Orthotopic Pancreatic Tumor Growth and Intra-abdominal Metastases

Pancreatic cancer is highly lethal. New diagnostic and treatment modalities are desperately needed. We report here that an expanded porphyrin, cyclo[8]pyrrole (CP), with a high extinction coefficient (89.16 L/g·cm) within the second near-infrared window (NIR-II), may be formulated with an αvβ3-specific targeting peptide, cyclic-Arg-Gly-Asp (cRGD), to form cRGD-CP nanoparticles (cRGD-CPNPs) with promising NIR-II photothermal (PT) therapeutic and photoacoustic (PA) imaging properties. Studies with a ring-array PA tomography system, coupled with analysis of control nanoparticles lacking a targeting element (CPNPs), revealed that cRGD conjugation promoted the delivery of the NPs through abnormal vessels around the tumor to the solid tumor core. This proved true in both subcutaneous and orthotopic pancreatic tumor mice models, as confirmed by immunofluorescent studies. In combination with NIR-II laser photoirradiation, the cRGD-CPNPs provided near-baseline tumor growth inhibition through PTT both in vitro and in vivo. Notably, the combination of the present cRGD-CPNPs and photoirradiation was found to inhibit intra-abdominal metastases in an orthotopic pancreatic tumor mouse model. The cRGD-CPNPs also displayed good biosafety profiles, as inferred from PA tomography, blood analyses, and H&E staining. They thus appear promising for use in combined PA imaging and PT therapeutic treatment of pancreatic cancer

Multivariate analysis of associations between clinical sequencing and outcome in glioblastoma

BACKGROUND: Many factors impact survival in patients with glioblastoma, including age, Karnofsky Performance Status, postoperative chemoradiation, METHODS: We utilized a widely available diagnostic platform (FoundationOne CDx) to perform high-throughput next-generation sequencing on 185 patients with newly diagnosed glioblastoma in our tertiary care center. We performed multivariate analysis to control for clinical parameters with known impact on survival to elucidate the independent prognostic value of prevalent mutant genes and the independent impact of gross total resection. RESULTS: When controlling for factors with known prognostic significance including CONCLUSIONS: This study verifies the independent prognostic value of several mutant genes in glioblastoma. Six commonly found mutant genes were associated with improved survival when gross total resection was achieved. Thus, even when accounting for known predictors of survival and multiple mutant gene comparisons, extent of resection continues to be strongly associated with survival