Mitigating Shortcuts in Language Models with Soft Label Encoding

Recent research has shown that large language models rely on spurious correlations in the data for natural language understanding (NLU) tasks. In this work, we aim to answer the following research question: Can we reduce spurious correlations by modifying the ground truth labels of the training data? Specifically, we propose a simple yet effective debiasing framework, named Soft Label Encoding (SoftLE). We first train a teacher model with hard labels to determine each sample's degree of relying on shortcuts. We then add one dummy class to encode the shortcut degree, which is used to smooth other dimensions in the ground truth label to generate soft labels. This new ground truth label is used to train a more robust student model. Extensive experiments on two NLU benchmark tasks demonstrate that SoftLE significantly improves out-of-distribution generalization while maintaining satisfactory in-distribution accuracy

Ceramide in cerebrovascular diseases

Ceramide, a bioactive sphingolipid, serves as an important second messenger in cell signal transduction. Under stressful conditions, it can be generated from de novo synthesis, sphingomyelin hydrolysis, and/or the salvage pathway. The brain is rich in lipids, and abnormal lipid levels are associated with a variety of brain disorders. Cerebrovascular diseases, which are mainly caused by abnormal cerebral blood flow and secondary neurological injury, are the leading causes of death and disability worldwide. There is a growing body of evidence for a close connection between elevated ceramide levels and cerebrovascular diseases, especially stroke and cerebral small vessel disease (CSVD). The increased ceramide has broad effects on different types of brain cells, including endothelial cells, microglia, and neurons. Therefore, strategies that reduce ceramide synthesis, such as modifying sphingomyelinase activity or the rate-limiting enzyme of the de novo synthesis pathway, serine palmitoyltransferase, may represent novel and promising therapeutic approaches to prevent or treat cerebrovascular injury-related diseases

Explainability for Large Language Models: A Survey

Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. However, their internal mechanisms are still unclear and this lack of transparency poses unwanted risks for downstream applications. Therefore, understanding and explaining these models is crucial for elucidating their behaviors, limitations, and social impacts. In this paper, we introduce a taxonomy of explainability techniques and provide a structured overview of methods for explaining Transformer-based language models. We categorize techniques based on the training paradigms of LLMs: traditional fine-tuning-based paradigm and prompting-based paradigm. For each paradigm, we summarize the goals and dominant approaches for generating local explanations of individual predictions and global explanations of overall model knowledge. We also discuss metrics for evaluating generated explanations, and discuss how explanations can be leveraged to debug models and improve performance. Lastly, we examine key challenges and emerging opportunities for explanation techniques in the era of LLMs in comparison to conventional machine learning models

Soft Computing and Decision Support System for Software Process Improvement: A Systematic Literature Review

Software project development is very crucial, and measuring the exact cost and effort of development is becoming tedious and challenging. Organizations are trying to wind up their project of software development within the agreed budget and schedule successfully. Traditional practices are inadequate to achieve the current needs of the software industry. Underestimation and overestimation of software development effort lead to financial implications in the form of resources, cost of staffing, and budget of developing the software project. Soft computing (SC) approaches and tools deliver an addition of techniques for anticipating resistance to the deception, defect, incomplete truth for traceability and ambiguity, low arrangement cost, and strength. A large amount of SC approaches is prevailing in the literature to accomplish way-out to difficulties precisely, practically, and speedily. The approaches of SC can give better prediction, high performance, and dynamic behavior. SC deals with computational intelligence which integrates the concept of agent paradigm and SC. The proposed study presents a systematic literature review (SLR) of the approaches, tools, and techniques of SC used in the literature. The study presented a comprehensive review by searching the defined keywords in the popular libraries, filtered the paper, and obtained most relevant papers. After the selection of the papers, the quality assessment process of the included papers has been done in order to determine the relevancy of the papers. The study will help researchers in the area of research to devise novel ideas and solutions to overcome the existing issue on the basis of this study as evidence of the literatur