Extending Multi-modal Contrastive Representations

Multi-modal contrastive representation (MCR) of more than three modalities is critical in multi-modal learning. Although recent methods showcase impressive achievements, the high dependence on large-scale, high-quality paired data and the expensive training costs limit their further development. Inspired by recent C-MCR, this paper proposes Extending Multimodal Contrastive Representation (Ex-MCR), a training-efficient and paired-data-free method to flexibly learn unified contrastive representation space for more than three modalities by integrating the knowledge of existing MCR spaces. Specifically, Ex-MCR aligns multiple existing MCRs into the same based MCR, which can effectively preserve the original semantic alignment of the based MCR. Besides, we comprehensively enhance the entire learning pipeline for aligning MCR spaces from the perspectives of training data, architecture, and learning objectives. With the preserved original modality alignment and the enhanced space alignment, Ex-MCR shows superior representation learning performance and excellent modality extensibility. To demonstrate the effectiveness of Ex-MCR, we align the MCR spaces of CLAP (audio-text) and ULIP (3D-vision) into the CLIP (vision-text), leveraging the overlapping text and image modality, respectively. Remarkably, without using any paired data, Ex-MCR learns a 3D-image-text-audio unified contrastive representation, and it achieves state-of-the-art performance on audio-visual, 3D-image, audio-text, visual-text retrieval, and 3D object classification tasks. More importantly, extensive qualitative results further demonstrate the emergent semantic alignment between the extended modalities (e.g., audio and 3D), which highlights the great potential of modality extensibility.Comment: Our code is available at https://github.com/MCR-PEFT/Ex-MC

A novel nomogram for adult primary perihilar cholangiocarcinoma and considerations concerning lymph node dissection

ObjectiveTo construct a reliable nomogram available online to predict the postoperative survival of patients with perihilar cholangiocarcinoma.MethodsData from 1808 patients diagnosed with perihilar cholangiocarcinoma between 2004 and 2015 were extracted from the National Cancer Institute Surveillance, Epidemiology, and End Results (SEER) database. They were randomly divided into training and validation sets. The nomogram was established by machine learning and Cox model. The discriminant ability and prediction accuracy of the nomogram were evaluated by concordance index (C-index), receiver operator characteristic (ROC) curve and calibration curve. Kaplan-Meier curves show the prognostic value of the associated risk factors and classification system.ResultsMachine learning and multivariate Cox risk regression model showed that sex, age, tumor differentiation, primary tumor stage(T), lymph node metastasis(N), TNM stage, surgery, radiation, chemotherapy, lymph node dissection were associated with the prognosis of perihilar cholangiocarcinoma patients relevant factors (P < 0.05). A novel nomogram was established. The calibration plots, C-index and ROC curve for predictions of the 1-, 3-, and 5-year OS were in excellent agreement. In patients with stage T1 and N0 perihilar cholangiocarcinoma, the prognosis of ≥4 lymph nodes dissected was better than that of 1- 3 lymph nodes dissected (P < 0.01).ConclusionThe nomogram prognostic prediction model can provide a reference for evaluating the prognosis and survival rate of patients with perihilar cholangiocarcinoma. Patients with stage T1 and N0 perihilar cholangiocarcinoma have more benefits by increasing the number of lymph node dissection

Wetting properties of cosmetic polymeric solutions on hair tresses

© 2016 Elsevier B.V.The objective of the present work is to investigate wetting of hair tresses with the solutions of two polyacrylate polymers broadly used in cosmetic products. Wetting properties of the neutralized Aculyn-22™ (A22) and Aculyn-33™ (A33) polymer solutions on dry hair tresses are studied. Wetting behaviour on the dry undamaged hair tresses is drastically different between the two polymers and, in a first approximation, not directly linked with their bulk rheology. In the case of A22 the droplet spreads and remains on the tress after spreading for at least half an hour, during which it slowly evaporates and possibly penetrates inside the hair. For A33 fast penetration of the droplet inside the hair tress is observed when the advancing contact angle reaches a critical value of about 60°. It can be attributed to the so-called Cassie-Wenzel wetting transition, in which the liquid starts to penetrate inside the hair array

Connecting Multi-modal Contrastive Representations

Multi-modal Contrastive Representation learning aims to encode different modalities into a semantically aligned shared space. This paradigm shows remarkable generalization ability on numerous downstream tasks across various modalities. However, the reliance on massive high-quality data pairs limits its further development on more modalities. This paper proposes a novel training-efficient method for learning MCR without paired data called Connecting Multi-modal Contrastive Representations (C-MCR). Specifically, given two existing MCRs pre-trained on (A, B) and (B, C) modality pairs, we project them to a new space and use the data from the overlapping modality B to aligning the two MCRs in the new space. Meanwhile, since the modality pairs (A, B) and (B, C) are already aligned within each MCR, the connection learned by overlapping modality can also be transferred to non-overlapping modality pair (A, C). To unleash the potential of C-MCR, we further introduce a semantic-enhanced inter- and intra-MCR connection method. We first enhance the semantic consistency and completion of embeddings across different modalities for more robust alignment. Then we utilize the inter-MCR alignment to establish the connection, and employ the intra-MCR alignment to better maintain the connection for inputs from non-overlapping modalities. To demonstrate the effectiveness of C-MCR, we connect CLIP and CLAP via texts to derive audio-visual representations, and integrate CLIP and ULIP via images for 3D-language representations. Remarkably, without using any paired data, C-MCR for audio-visual achieves state-of-the-art performance on audio-image retrieval, audio-visual source localization, and counterfactual audio-image recognition tasks. Furthermore, C-MCR for 3D-language also attains advanced zero-shot 3D point cloud classification accuracy on ModelNet40.Comment: NeurIPS 202

Strong magnon-magnon coupling in an ultralow damping all-magnetic-insulator heterostructure

Magnetic insulators such as yttrium iron garnets (YIGs) are of paramount importance for spin-wave or magnonic devices as their ultralow damping enables ultralow power dissipation that is free of Joule heating, exotic magnon quantum state, and coherent coupling to other wave excitations. Magnetic insulator heterostructures bestow superior structural and magnetic properties and house immense design space thanks to the strong and engineerable exchange interaction between individual layers. To fully unleash their potential, realizing low damping and strong exchange coupling simultaneously is critical, which often requires high quality interface. Here, we show that such a demand is realized in an all-insulator thulium iron garnet (TmIG)/YIG bilayer system. The ultralow dissipation rates in both YIG and TmIG, along with their significant spin-spin interaction at the interface, enable strong and coherent magnon-magnon coupling with a benchmarking cooperativity value larger than the conventional ferromagnetic metal-based heterostructures. The coupling strength can be tuned by varying the magnetic insulator layer thickness and magnon modes, which is consistent with analytical calculations and micromagnetic simulations. Our results demonstrate TmIG/YIG as a novel platform for investigating hybrid magnonic phenomena and open opportunities in magnon devices comprising all-insulator heterostructures.Comment: 45 pages, 18 figures, and 2 table

Seismic Risk Assessment of Spatially Distributed Levee System in the Sacramento-San Joaquin Delta

The approximately 1,100 miles of levees in the Sacramento-San Joaquin Delta is critical to aquatic and terrestrial habitat, agriculture, California’s water supply and distribution system, and other infrastructure investments, and the levee system protects them from flooding and salt water intrusion. However, the levee system is threatened by a variety of hazards. Land due to oxidation of the rich Delta peat soils, and due to sea level risk act together to effectively increase the levee hydraulic loading. Consolidation of peat soils beneath levees can lead to their continued settlement over time. Delta levees are also threatened by potential sudden shocks from floods events and earthquakes. Numerous advances with greater proliferation and more sophisticated methods of risk assessments have been made since the most recent risk study of the Delta was completed. Therefore, assessing multi-hazard risks of the Delta levee system by leveraging newly available data and knowledge is of great importance for decision makers to implement improvements in response to those long-term and short-term stressors.This study primarily focuses on seismic risk assessment of Bacon Island in the central Delta. The seismic capacity, demand, spatial correlations of levee systems, and system reliability analysis are four essential components throughout the seismic risk assessment. Newly available LiDAR, bathymetry data, geotechnical site investigation results, and measurements from advanced geophysical tests significantly facilitate determining geometry, soil stratigraphy/layering, and soil property of levees. Consequently, the levee fragility functions which reflect the system seismic capacity are developed from a large number of time-series nonlinear finite element simulations using OpenSees. An overview of updated probabilistic seismic hazard analysis results for the Delta region is discussed. Moreover, an algorithm for selecting a subset of events for hazard-consistent analysis of spatially distributed infrastructures is introduced, and performed to analyze the regional probabilistic seismic hazard analysis of the Bacon Island levee system, which quantifies seismic demand of the levees. The correlation functions of capacity are derived based on field geophysical measurements and geo-statistics analysis. Furthermore, the system reliability analysis using level crossing statistics method is implemented to assess seismic risk for Bacon Island levees based on the developed levee fragility, correlation lengths, and selected event subset