Sentiment Word Aware Multimodal Refinement for Multimodal Sentiment Analysis with ASR Errors

Multimodal sentiment analysis has attracted increasing attention and lots of models have been proposed. However, the performance of the state-of-the-art models decreases sharply when they are deployed in the real world. We find that the main reason is that real-world applications can only access the text outputs by the automatic speech recognition (ASR) models, which may be with errors because of the limitation of model capacity. Through further analysis of the ASR outputs, we find that in some cases the sentiment words, the key sentiment elements in the textual modality, are recognized as other words, which makes the sentiment of the text change and hurts the performance of multimodal sentiment models directly. To address this problem, we propose the sentiment word aware multimodal refinement model (SWRM), which can dynamically refine the erroneous sentiment words by leveraging multimodal sentiment clues. Specifically, we first use the sentiment word position detection module to obtain the most possible position of the sentiment word in the text and then utilize the multimodal sentiment word refinement module to dynamically refine the sentiment word embeddings. The refined embeddings are taken as the textual inputs of the multimodal feature fusion module to predict the sentiment labels. We conduct extensive experiments on the real-world datasets including MOSI-Speechbrain, MOSI-IBM, and MOSI-iFlytek and the results demonstrate the effectiveness of our model, which surpasses the current state-of-the-art models on three datasets. Furthermore, our approach can be adapted for other multimodal feature fusion models easily. Data and code are available at https://github.com/albertwy/SWRM.Comment: Findings of ACL 202

Research on the Artificial Intelligence Attribution Analysis and the Reasons for Decline of Physical Fitness of College Students in the Tropical Area under the Background of "Sunshine Sports"

In 2010, the research team conducted a survey on the physical quality of 1600 college students in Hainan Province, and analyzed the changes in the physical quality of college students in Hainan from 2000 to 2010, and learned about the impact of sunshine sports on the physical quality of college students in Hainan Province. Based on this, a targeted intervention was proposed, which can provide reference for the sports workers and decision-makers in Hainan Province and improve the physical quality of students

Traffic volume and load data measurement using a portable weigh in motion system: A case study

AbstractTraditionally, traffic loading characteristics are collected for pavement design and performance prediction purposes using permanent roadside weigh-in-motion (WIM) stations. However, high installation and maintenance costs associated with these permanent WIM stations dictate that their deployment be mostly limited to major highways, such as the interstate network. Quite often however, pavement damage on high volume rural highways with heavy truck proportions is more severe than anticipated, and there is no effective way of quantifying the traffic loading on these highways. Therefore, this study was conducted to evaluate the potential application of portable WIM systems as a means for bringing the WIM technology to these high volume rural highways. A portable WIM unit was deployed in the Texas overweight corridor in Hidalgo County (Pharr District) near the USA-Mexico border on highway FM 1016 for collecting traffic data for a minimum of three weeks in each direction. The collected traffic data were analyzed to generate traffic parameters such as volume, load spectra, and overloading information both in terms of the gross vehicle weight (GVW) and axle weight. The computed traffic parameters were successful in partially explaining some of the existing pavement conditions on this highway. Overall, the study findings indicated that the portable WIM unit can be used as a convenient and cost-effective means for collecting reliable traffic information for design, analysis, and monitoring purposes. However, proper in-situ calibration of the portable WIM unit at each site is imperative prior to any real-time traffic data collection

Dark against luminous matter around isolated central galaxies: a comparative study between modern surveys and Illustris-TNG

Based on independent shear measurements using the DECaLS/DR8 imaging data, we measure the weak lensing signals around isolated central galaxies (ICGs) from SDSS/DR7 at $z\sim0.1$. The projected stellar mass density profiles of surrounding satellite galaxies are further deduced, using photometric sources from the Hyper Suprime-Cam (HSC) survey (pDR3). The signals of ICGs $+$ their extended stellar halos are taken from Wang et al.(2021). All measurements are compared with predictions by the Illustris-TNG300-1 simulation. We find, overall, a good agreement between observation and TNG300. In particular, a correction to the stellar mass of massive observed ICGs is applied based on the calibration of He et al.(2013), which brings a much better agreement with TNG300 predicted lensing signals at $\log_{10}M_\ast/M_\odot>11.1$. In real observation, red ICGs are hosted by more massive dark matter halos, have more satellites and more extended stellar halos than blue ICGs at fixed stellar mass. However, in TNG300 there are more satellites around blue ICGs at fixed stellar mass, and the outer stellar halos of red and blue ICGs are similar. The stellar halos of TNG galaxies are more extended compared with real observed galaxies, especially for blue ICGs with $\log_{10}M_\ast/M_\odot>10.8$. We find the same trend for TNG100 galaxies and for true halo central galaxies. The tensions between TNG and real galaxies might indicate that satellite disruptions are stronger in TNG. In both TNG300 and observation, satellites approximately trace the underlying dark matter distribution beyond $0.1R_{200}$, but the fraction of total stellar mass in TNG300 does not show the same radial distribution as real galaxies.Comment: 28 pages, 12 figure