Improving Long Document Topic Segmentation Models With Enhanced Coherence Modeling

Topic segmentation is critical for obtaining structured documents and improving downstream tasks such as information retrieval. Due to its ability of automatically exploring clues of topic shift from abundant labeled data, recent supervised neural models have greatly promoted the development of long document topic segmentation, but leaving the deeper relationship between coherence and topic segmentation underexplored. Therefore, this paper enhances the ability of supervised models to capture coherence from both logical structure and semantic similarity perspectives to further improve the topic segmentation performance, proposing Topic-aware Sentence Structure Prediction (TSSP) and Contrastive Semantic Similarity Learning (CSSL). Specifically, the TSSP task is proposed to force the model to comprehend structural information by learning the original relations between adjacent sentences in a disarrayed document, which is constructed by jointly disrupting the original document at topic and sentence levels. Moreover, we utilize inter- and intra-topic information to construct contrastive samples and design the CSSL objective to ensure that the sentences representations in the same topic have higher similarity, while those in different topics are less similar. Extensive experiments show that the Longformer with our approach significantly outperforms old state-of-the-art (SOTA) methods. Our approach improve $F_1$ of old SOTA by 3.42 (73.74 -> 77.16) and reduces $P_k$ by 1.11 points (15.0 -> 13.89) on WIKI-727K and achieves an average relative reduction of 4.3% on $P_k$ on WikiSection. The average relative $P_k$ drop of 8.38% on two out-of-domain datasets also demonstrates the robustness of our approach.Comment: Accepted by EMNLP 2023. Codes is available at https://github.com/alibaba-damo-academy/SpokenNLP

Fatigue Assessment of Traffic Signal Mast Arms based on Field Test Data under Natural Wind Gusts

In recent years, several states including Missouri, Wyoming, California, and Texas experienced fracture failures of traffic signal mast arms. Almost all the failures are associated with the propagation of defects or cracks. It is therefore imperative to evaluate existing mast arms using a simple yet accurate procedure. A statistical methodology is proposed to predict the fatigue life of signal mast arm structures on the basis of field-measured strain data. The annual occurrence of various stress levels is determined using the historical wind speed data in the vicinity of a mast arm structure and the strain readings of the structure under specific wind gusts. For each stress level, the crack initiation and propagation lives are estimated with the strain-life approach and the Paris crack-growth-rate model. They are combined to account for variable stresses by means of Miner\u27s rule and the root-mean-square model, respectively. The stress concentration factor around the arm-post connection is determined using a finite element model. The parameters in the life prediction models are determined with ASTM flat tension and compact tension tests. The proposed methodology was applied to a 12.8-m (42-ft) long octagonal mast arm and a 16.5-m (54-ft) long circular mast arm in Missouri. It is concluded that signal structures in perfect condition will not crack under natural wind gusts during their service life. However, the 16.5-m-long arm is likely to be vulnerable to tiny defects around the weld connection, but the 12.8-m-long arm is safe unless a visible crack exists

Modality-invariant and Specific Prompting for Multimodal Human Perception Understanding

Understanding human perceptions presents a formidable multimodal challenge for computers, encompassing aspects such as sentiment tendencies and sense of humor. While various methods have recently been introduced to extract modality-invariant and specific information from diverse modalities, with the goal of enhancing the efficacy of multimodal learning, few works emphasize this aspect in large language models. In this paper, we introduce a novel multimodal prompt strategy tailored for tuning large language models. Our method assesses the correlation among different modalities and isolates the modality-invariant and specific components, which are then utilized for prompt tuning. This approach enables large language models to efficiently and effectively assimilate information from various modalities. Furthermore, our strategy is designed with scalability in mind, allowing the integration of features from any modality into pretrained large language models. Experimental results on public datasets demonstrate that our proposed method significantly improves performance compared to previous methods

Ditto: A Simple and Efficient Approach to Improve Sentence Embeddings

Prior studies diagnose the anisotropy problem in sentence representations from pre-trained language models, e.g., BERT, without fine-tuning. Our analysis reveals that the sentence embeddings from BERT suffer from a bias towards uninformative words, limiting the performance in semantic textual similarity (STS) tasks. To address this bias, we propose a simple and efficient unsupervised approach, Diagonal Attention Pooling (Ditto), which weights words with model-based importance estimations and computes the weighted average of word representations from pre-trained models as sentence embeddings. Ditto can be easily applied to any pre-trained language model as a postprocessing operation. Compared to prior sentence embedding approaches, Ditto does not add parameters nor requires any learning. Empirical evaluations demonstrate that our proposed Ditto can alleviate the anisotropy problem and improve various pre-trained models on STS tasks.Comment: 8 pages, accepted by EMNLP 2023 short paper, the source code can be found at https://github.com/alibaba-damo-academy/SpokenNLP/tree/main/ditt

Forensic Investigation of Failed Mast Arms of Traffic Signal Supported Structures

In Missouri, 11 traffic signal mast arms fractured at the arm-post weld connection in 7 years. To reduce this fatigue failure, the Missouri Department of Transportation developed a fatigue-resistant weld profile that increases the weld leg and reduces the slope of the weld at the toe. This study investigated causes of the failed arms, compared performance of new and old weld profiles, and suggested retrofitting measures for further investigation. The scope included a metallurgical investigation of one failed field mast arm, laboratory fatigue testing of five prototype mast arms (two new and three old profiles), and laboratory failure analysis of one arm tested to cracking. Metallographic and fractographic analyses indicated that the fatigue crack in the failed mast arm initiates near the weld toe of the arm due to undercutting, creating a sharp local toe angle. Location of undercutting at the heat-affected zone of the base material, where the material is softest, further contributed to early fatigue failure. Tests showed that the new weld profile does not consistently increase fatigue strength. Premature fracture surfaces of one tested arm indicated that the fatigue cracks initiate in an area at the weld toe as observed in the failed mast arm. Therefore, changing the weld profile alone is unlikely to increase mast arm fatigue life. Pinning the weldment surface at the weld toe of mast arms is suggested to increase the life of mast arms

Determining the Quantitative Threshold of High-Frequency Oscillation Distribution to Delineate the Epileptogenic Zone by Automated Detection

Objective: We proposed an improved automated high frequency oscillations (HFOs) detector that could not only be applied to various intracranial electrodes, but also automatically remove false HFOs caused by high-pass filtering. We proposed a continuous resection ratio of high order HFO channels and compared this ratio with each patient's post-surgical outcome, to determine the quantitative threshold of HFO distribution to delineate the epileptogenic zone (EZ).Methods: We enrolled a total of 43 patients diagnosed with refractory epilepsy. The patients were used to optimize the parameters for SEEG electrodes, to test the algorithm for identifying false HFOs, and to calculate the continuous resection ratio of high order HFO channels. The ratio can be used to determine a quantitative threshold to locate the epileptogenic zone.Results: Following optimization, the sensitivity, and specificity of our detector were 66.84 and 73.20% (ripples) and 69.76 and 66.13% (fast ripples, FRs), respectively. The sensitivity and specificity of our algorithm for removing false HFOs were 76.82 and 94.54% (ripples) and 72.55 and 94.87% (FRs), respectively. The median of the continuous resection ratio of high order HFO channels in patients with good surgical outcomes, was significantly higher than in patients with poor outcome, for both ripples and FRs (P < 0.05 ripples and P < 0.001 FRs).Conclusions: Our automated detector has the advantage of not only applying to various intracranial electrodes but also removing false HFOs. Based on the continuous resection ratio of high order HFO channels, we can set the quantitative threshold for locating epileptogenic zones

Microblog opinion evolution model

Retweeting is an important behavior on Twitter, indicating the behavior that users repost microblogs of their friends. While much work has been conducted for mining textual content that users generate or analyzing the social network structure, few publications systematically study the underlying mechanism of the retweeting behaviors. In this project, a new opinion evolution model is put forward. Based on several important observations, this model is developed to predict users’ view dissemination in a particular group. Several comparison Experiments have been done to show that the sensitivity as well as activity has a role to promote the dissemination of information.  Bachelor of Engineering (Computer Engineering