Sentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors

Multimodal sentiment analysis has attracted increasing attention and lots of models have been proposed. However, the performance of the state-of-the-art models decreases sharply when they are deployed in the real world. We find that the main reason is that real-world applications can only access the text outputs by the automatic speech recognition (ASR) models, which may be with errors because of the limitation of model capacity. Through further analysis of the ASR outputs, we find that in some cases the sentiment words, the key sentiment elements in the textual modality, are recognized as other words, which makes the sentiment of the text change and hurts the performance of multimodal sentiment models directly. To address this problem, we propose the sentiment word aware multimodal refinement model (SWRM), which can dynamically refine the erroneous sentiment words by leveraging multimodal sentiment clues. Specifically, we first use the sentiment word position detection module to obtain the most possible position of the sentiment word in the text and then utilize the multimodal sentiment word refinement module to dynamically refine the sentiment word embeddings. The refined embeddings are taken as the textual inputs of the multimodal feature fusion module to predict the sentiment labels. We conduct extensive experiments on the real-world datasets including MOSI-Speechbrain, MOSI-IBM, and MOSI-iFlytek and the results demonstrate the effectiveness of our model, which surpasses the current state-of-the-art models on three datasets. Furthermore, our approach can be adapted for other multimodal feature fusion models easily. Data and code are available at https://github.com/albertwy/SWRM.Comment: Findings of ACL 202

Editorial: Advances in deep learning methods for medical image analysis

The rapid development of artificial intelligence (AI) technology is leading many innovations in the medical field and is playing a major role in establishing objective, consistent, and efficient medical environments with large-scale data. Deep learning represented by convolutional neural networks has achieved remarkable performance improvement in medical image processing fields such as image segmentation, registration, and enhancement. Furthermore, AI technology with deep learning is pioneering medical applications, such as lesion detection, differential diagnosis, disease prognosis, and surgical planning. More advanced AI technologies, such as transformers with self-attention mechanisms, allowing for learning global dependencies, have been widely applied, which further enhanced the capability of deep learning to analyze medical images. However, despite the remarkable advances in deep learning, many challenges remain. For example, when training data are biased or incomplete, deep learning models may fail to achieve the good generalization capability required to solve real-world problems. In addition, the limitations of deep learning models in interpreting results, and misunderstandings of their intended uses and hypotheses make it difficult for AI to gain trust in healthcare settings. In this regard, disease-specific neural networks, generalized learning methods, high-quality training data, and external evaluation based on testable hypotheses can ensure the reliability of medical AI technologies for humans

Targeted next-generation sequencing of dedifferentiated chondrosarcoma in the skull base reveals combined TP53 and PTEN mutations with increased proliferation index, an implication for pathogenesis

Dedifferentiated chondrosarcoma (DDCS) is a rare disease with a dismal prognosis. DDCS consists of two morphologically distinct components: the cartilaginous and noncartilaginous components. Whether the two components originate from the same progenitor cells has been controversial. Recurrent DDCS commonly displays increased proliferation compared with the primary tumor. However, there is no conclusive explanation for this mechanism. In this paper, we present two DDCSs in the sellar region. Patient 1 exclusively exhibited a noncartilaginous component with a TP53 frameshift mutation in the pathological specimens from the first surgery. The tumor recurred after radiation therapy with an exceedingly increased proliferation index. Targeted next-generation sequencing (NGS) revealed the presence of both a TP53 mutation and a PTEN deletion in the cartilaginous and the noncartilaginous components of the recurrent tumor. Fluorescence in situ hybridization and immunostaining confirmed reduced DNA copy number and protein levels of the PTEN gene as a result of the PTEN deletion. Patient 2 exhibited both cartilaginous and noncartilaginous components in the surgical specimens. Targeted NGS of cells from both components showed neither TP53 nor PTEN mutations, making Patient 2 a naïve TP53 and PTEN control for comparison. In conclusion, additional PTEN loss in the background of the TP53 mutation could be the cause of increased proliferation capacity in the recurrent tumor

Clinical, neurophysiological evaluation and genetic features of axonal Charcot–Marie–Tooth disease in a Chinese family

Charcot–Marie–Tooth disease (CMT) is a group of inherited peripheral neuropathies related to variants in the mitochondrial transfer RNA (mt-tRNAval) gene. Here, we report a Chinese family harboring the m.1661A>G variant in the mt-tRNAval gene. Clinical evaluation, neuroelectrodiagnostic testing, and nerve biopsy were performed on four affected family members. Weakness, spasms, and pain in the limbs (especially in the lower limbs) were the main complaints of the proband. Physical examination revealed atrophy and weakness in the distal limbs, increased muscle tone, and hyperreflexia in four limbs. Neuroelectrodiagnostic tests and nerve biopsy supported an axonal polyneuropathy. This study furthers the understanding of phenotype diversity caused by variants in the mt-tRNAval gene in CMT