Explainability for Large Language Models: A Survey

Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. However, their internal mechanisms are still unclear and this lack of transparency poses unwanted risks for downstream applications. Therefore, understanding and explaining these models is crucial for elucidating their behaviors, limitations, and social impacts. In this paper, we introduce a taxonomy of explainability techniques and provide a structured overview of methods for explaining Transformer-based language models. We categorize techniques based on the training paradigms of LLMs: traditional fine-tuning-based paradigm and prompting-based paradigm. For each paradigm, we summarize the goals and dominant approaches for generating local explanations of individual predictions and global explanations of overall model knowledge. We also discuss metrics for evaluating generated explanations, and discuss how explanations can be leveraged to debug models and improve performance. Lastly, we examine key challenges and emerging opportunities for explanation techniques in the era of LLMs in comparison to conventional machine learning models

Critical Role of AKT in Myeloma-induced Osteoclast Formation and Osteolysis

Abnormal osteoclast formation and osteolysis are the hallmarks of multiple myeloma (MM) bone disease, yet the underlying molecular mechanisms are incompletely understood. Here, we show that the AKT pathway was up-regulated in primary bone marrow monocytes (BMM) from patients with MM, which resulted in sustained high expression of the receptor activator of NF-κB (RANK) in osteoclast precursors. The up-regulation of RANK expression and osteoclast formation in the MM BMM cultures was blocked by AKT inhibition. Conditioned media from MM cell cultures activated AKT and increased RANK expression and osteoclast formation in BMM cultures. Inhibiting AKT in cultured MM cells decreased their growth and ability to promote osteoclast formation. Of clinical significance, systemic administration of the AKT inhibitor LY294002 blocked the formation of tumor tissues in the bone marrow cavity and essentially abolished the MM-induced osteoclast formation and osteolysis in SCID mice. The level of activating transcription factor 4 (ATF4) protein was up-regulated in the BMM cultures from multiple myeloma patients. Adenoviral overexpression of ATF4 activated RANK expression in osteoclast precursors. These results demonstrate a new role of AKT in the MM promotion of osteoclast formation and bone osteolysis through, at least in part, the ATF4-dependent up-regulation of RANK expression in osteoclast precursors

3D bioprinted hydrogel/polymer scaffold with factor delivery and mechanical support for growth plate injury repair

Introduction: Growth plate injury is a significant challenge in clinical practice, as it could severely affect the limb development of children, leading to limb deformity. Tissue engineering and 3D bioprinting technology have great potential in the repair and regeneration of injured growth plate, but there are still challenges associated with achieving successful repair outcomes.Methods: In this study, GelMA hydrogel containing PLGA microspheres loaded with chondrogenic factor PTH(1–34) was combined with BMSCs and Polycaprolactone (PCL) to develop the PTH(1–34)@PLGA/BMSCs/GelMA-PCL scaffold using bio-3D printing technology.Results: The scaffold exhibited a three-dimensional interconnected porous network structure, good mechanical properties, biocompatibility, and was suitable for cellchondrogenic differentiation. And a rabbit model of growth plate injury was appliedto validate the effect of scaffold on the repair of injured growth plate. The resultsshowed that the scaffold was more effective than injectable hydrogel in promotingcartilage regeneration and reducing bone bridge formation. Moreover, the addition ofPCL to the scaffold provided good mechanical support, significantly reducing limbdeformities after growth plate injury compared with directly injected hydrogel.Discussion: Accordingly, our study demonstrates the feasibility of using 3D printed scaffolds for treating growth plate injuries and could offer a new strategy for the development of growth plate tissue engineering therapy

Noise Separation from the Weak Signal Chaotic Detection System

The traditional weak signal chaotic detection system still restricts some technical issues in the situation of the signal with noise, such as poor denoising ability and low detection precision. In this paper, we propose a novel weak signal chaotic detection system based on an improved wavelet transform algorithm. First, the traditional wavelet transform algorithm domain variables have been transformed and discretized to eliminate the redundant transform. Then, based on the discrete optimization, the wavelet coefficients have been optimized by threshold compromise strategy. The improved wavelet transform algorithm is applied in the weak signal chaotic detection system. The noise signal after finite discrete processing is treated as a perturbation of cycle power and put into a chaotic system for detecting weak signal under the noise conditions. The simulation experiments show that the proposed improved wavelet transform algorithm has a better denoising effect than the traditional wavelet transform algorithm. Moreover, the improved algorithm shows better accuracy and higher robustness in the weak signal chaotic detection system

Improvement of Typhoon Intensity Forecasting by Using a Novel Spatio-Temporal Deep Learning Model

Typhoons can cause massive casualties and economic damage, and accurately predicting typhoon intensity has always been a hot topic both in theory and practice. In consideration with the spatial and temporal complexity of typhoons, machine learning methods have recently been applied in typhoon forecasting. In this paper, we attempt to improve typhoon intensity forecasting by treating it as a spatio-temporal problem in the deep learning field. In particular, we propose a novel typhoon intensity forecasting model named the Typhoon Intensity Spatio-temporal Prediction Network (TITP-Net). The proposed model takes multidimensional environmental variables and physical factors of typhoons into account and fully extracts the information from the datasets by capturing spatio-temporal dependencies with a spatial attention module, which includes two-dimensional and three-dimensional convolutional operations. A series of experiments with a comprehensive framework by using TITP-Net are conducted. The MAEs of the forecasts with 18, 24, 36 and 48 h lead time obtain a significant improvement by 7.02%, 6.53%, 6.25% and 5.37% compared with some existing deep learning models and dynamical models from official agencies

Asymmetry Between Positive and Negative Phases of the Pacific Meridional Mode: A Contributor to ENSO Transition Complexity

Abstract The Pacific Meridional Mode (PMM) plays a critical role in affecting El Niño‐Southern Oscillation (ENSO). This study examines the phase asymmetry of PMM events triggered by tropical and extratropical forcings, namely successive and stochastic events, respectively. It is shown that successive events exhibit negative asymmetry due to stronger trigger in the negative phase, while stochastic events display positive asymmetry due to stronger growth in the positive phase. The opposite phase asymmetry of two types of events respectively results in more frequent persistent La Niña events than El Niño events and more frequent episodic El Niño events than La Niña events, which increase ENSO transition complexity. This research provides a comprehensive understanding of PMM asymmetry and reconciles conflicting perspectives from previous studies. Additionally, the newly proposed contribution of positively asymmetric stochastic PMM events to more frequent episodic El Niño events in this study may enhance our comprehension of ENSO transition complexity

Improvement of Typhoon Intensity Forecasting by Using a Novel Spatio-Temporal Deep Learning Model

Typhoons can cause massive casualties and economic damage, and accurately predicting typhoon intensity has always been a hot topic both in theory and practice. In consideration with the spatial and temporal complexity of typhoons, machine learning methods have recently been applied in typhoon forecasting. In this paper, we attempt to improve typhoon intensity forecasting by treating it as a spatio-temporal problem in the deep learning field. In particular, we propose a novel typhoon intensity forecasting model named the Typhoon Intensity Spatio-temporal Prediction Network (TITP-Net). The proposed model takes multidimensional environmental variables and physical factors of typhoons into account and fully extracts the information from the datasets by capturing spatio-temporal dependencies with a spatial attention module, which includes two-dimensional and three-dimensional convolutional operations. A series of experiments with a comprehensive framework by using TITP-Net are conducted. The MAEs of the forecasts with 18, 24, 36 and 48 h lead time obtain a significant improvement by 7.02%, 6.53%, 6.25% and 5.37% compared with some existing deep learning models and dynamical models from official agencies

Dwindling vanadium in seawater during the early Cambrian, South China

Elemental vanadium (V), an essentially redox-sensitive metal in seawater, has had a significant impact on the understanding of the evolution of the atmosphere-ocean system throughout the history of the Earth. In fact, the geochemical cycle of V in early Cambrian seawater may have had an influence on the Chengjiang Biota in South China; however, it has not yet been well established. Given the authigenic vanadium accumulation is sensitive to the redox conditions of seawater, here, to constrain the geochemical cycle of V in seawater during the early Cambrian, the Mo, U and total organic carbon (TOC) distributions with high-resolution samples from both the outer shelf and slope facies (e.g., the Duoding and Longbizui sections), are applied to evaluate the redox conditions of ambient seawater. The Mo-U relationships indicate that the redox conditions of the mid-depth seawater evolved in a systematic way in South China, transitioning from an Fe-Mn reduction zone to anoxic/intermittently euxinic states and then to oxic conditions during the early Cambrian. As a consequence, the authigenic V enrichment, constrained by the marine redox conditions, was mainly controlled by the Fe-Mn particulate shuttle and the reduction and adsorption of organic matter in anoxic/euxinic conditions. However, the decoupling among V, Mo, U and TOC under anoxic/euxinic conditions suggests a dwindling vanadium concentration in the early Cambrian seawater of South China. The scavenging efficiency of V from seawater is much higher than those of Mo and U under anoxic/euxinic conditions. Ultimately, these trace elements (e.g., Mo, U, and especially V) in seawater could effectively be regulated and adjusted to a reasonable level under the widespread anoxic/euxinic conditions. The drawdown of trace elements in seawater might provide an earlystage preparation of the marine environment for the subsequent Chengjiang Biota