Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Model

Sentence Representation Learning (SRL) is a fundamental task in Natural Language Processing (NLP), with Contrastive learning of Sentence Embeddings (CSE) as the mainstream technique due to its superior performance. An intriguing phenomenon in CSE is the significant performance gap between supervised and unsupervised methods, even when their sentence encoder and loss function are the same. Previous works attribute this performance gap to differences in two representation properties (alignment and uniformity). However, alignment and uniformity only measure the results, which means they cannot answer "What happens during the training process that leads to the performance gap?" and "How can the performance gap be narrowed?". In this paper, we conduct empirical experiments to answer these "What" and "How" questions. We first answer the "What" question by thoroughly comparing the behavior of supervised and unsupervised CSE during their respective training processes. From the comparison, We observe a significant difference in fitting difficulty. Thus, we introduce a metric, called Fitting Difficulty Increment (FDI), to measure the fitting difficulty gap between the evaluation dataset and the held-out training dataset, and use the metric to answer the "What" question. Then, based on the insights gained from the "What" question, we tackle the "How" question by increasing the fitting difficulty of the training dataset. We achieve this by leveraging the In-Context Learning (ICL) capability of the Large Language Model (LLM) to generate data that simulates complex patterns. By utilizing the hierarchical patterns in the LLM-generated data, we effectively narrow the gap between supervised and unsupervised CSE.Comment: work in progres

Advance Directive and End-of-Life Care Preferences Among Nursing Home Residents in Wuhan, China: A Cross-Sectional Study

© 2014 AMDA - The Society for Post-Acute and Long-Term Care Medicine. Objectives: To describe Chinese nursing home residents' knowledge of advance directive (AD) and end-of-life care preferences and to explore the predictors of their preference for AD. Design: Population-based cross-sectional survey. Settings: Nursing homes (n= 31) in Wuhan, Mainland Southern China. Participants: Cognitively intact nursing home residents (n= 467) older than 60 years. Measures: Face-to-face questionnaire interviews were used to collect information on demographics, chronic diseases, life-sustaining treatment, AD, and other end-of-life care preferences. Results: Most (95.3%) had never heard of AD, and fewer than one-third (31.5%) preferred to make an AD. More than half (52.5%) would receive life-sustaining treatment if they sustained a life-threatening condition. Fewer than one-half (43.3%) chose doctors as the surrogate decision maker about life-sustaining treatment, whereas most (78.8%) nominated their eldest son or daughter as their proxy. More than half (58.2%) wanted to live and die in their present nursing homes. The significant independent predictors of AD preference included having heard of AD before (odds ratio [OR] 9.323), having definite answers of receiving (OR 3.433) or rejecting (OR 2.530) life-sustaining treatment, and higher Cumulative Illness Rating Scale score (OR 1.098). Conclusions: Most nursing home residents did not know about AD, and nearly one-third showed positive attitudes toward it. AD should be promoted in mainland China. Education of residents, the proxy decision maker, and nursing home staff on AD is very important. Necessary policy support, legislation, or practice guidelines about AD should be made with flexibility to respect nursing home residents' rights in mainland China