Ditto: A Simple and Efficient Approach to Improve Sentence Embeddings

Prior studies diagnose the anisotropy problem in sentence representations from pre-trained language models, e.g., BERT, without fine-tuning. Our analysis reveals that the sentence embeddings from BERT suffer from a bias towards uninformative words, limiting the performance in semantic textual similarity (STS) tasks. To address this bias, we propose a simple and efficient unsupervised approach, Diagonal Attention Pooling (Ditto), which weights words with model-based importance estimations and computes the weighted average of word representations from pre-trained models as sentence embeddings. Ditto can be easily applied to any pre-trained language model as a postprocessing operation. Compared to prior sentence embedding approaches, Ditto does not add parameters nor requires any learning. Empirical evaluations demonstrate that our proposed Ditto can alleviate the anisotropy problem and improve various pre-trained models on STS tasks.Comment: 8 pages, accepted by EMNLP 2023 short paper, the source code can be found at https://github.com/alibaba-damo-academy/SpokenNLP/tree/main/ditt

A Compound Fault Labeling and Diagnosis Method Based on Flight Data and BIT Record of UAV

In the process of Unmanned Aerial Vehicle (UAV) flight testing, plenty of compound faults exist, which could be composed of concurrent single faults or over-limit states alarmed by Built-In-Test (BIT) equipment. At present, there still lacks a suitable automatic labeling approach for UAV flight data, effectively utilizing the information of the BIT record. The performance of the originally employed flight data-driven fault diagnosis models based on machine learning needs to be improved as well. A compound fault labeling and diagnosis method based on actual flight data and the BIT record of the UAV during flight test phase is proposed, through labeling the flight data with compound fault modes corresponding to concurrent single faults recorded by the BIT system, and upgrading the original diagnosis model based on Gradient Boosting Decision Tree (GBDT) and Fully Convolutional Network (FCNN), to eXtreme Gradient Boosting (XGBoost), Light Gradient Boosting Machine (LightGBM) and modified Convolutional Neural Network (CNN). The experimental results based on actual test flight data show that the proposed method could effectively label the flight data and obtain a significant improvement in diagnostic performance, appearing to be practical in the UAV test flight process

Effect of Hot Deformation on PPB Precipitations and Microstructure in P/M Superalloy FGH96

HIPed FGH96 superalloy prepared by AA powder was used as the research object, and the PPB precipitations and microstructures before and after different hot deformations were studied. The results show that PPB precipitates with circular distribution can be seen in HIPed FGH96 superalloy, and the coarse strip prime γ′ phase with circular chain distribution is one of the main characters of PPB. After hot deformation, the PPB in FGH96 superalloy is deformed and broken, the recrystallization grains are refined and twisted, and the grain misorientation becomes smaller. With the increase of deformation temperature and hot deformation, and the decrease of deformation rate, the deformation and fragmentation of PPB precipitates are more significant, the size of the primary residual γ′ phase becomes smaller, and the volume becomes less. The increase of deformation temperature and hot deformation, and the decrease of deformation rate are beneficial to reduce or even eliminate PPB precipitations

The spatial distribution of traditional intangible cultural heritage medicine of China and its influencing factors

Abstract Intangible cultural heritage (ICH) of traditional Chinese medicine is a vivid testimony to the continuous inheritance of Chinese civilization. The study of ICH of traditional Chinese medicine from the perspective of the geographical environment, spatial relationships and diachronic development is of great significance for understanding objectively the reasons for the formation of spatial distribution patterns of ICH and the pathways and extent of its spread; as a consequence, such information can help us improve our understanding of the integrity and historical continuity of culture. From a geography perspective, this study seeks to explore the spatial patterns of traditional Chinese medicine ICH based on knowledge concerning the extent of spreading of traditional Chinese medicine ICH. At the same time, the spatial analysis technique ArcGIS is used to express the spatiotemporal development of traditional Chinese medicine ICH in a simple, clear and visual way. Also, the factors influencing the spatial distribution of traditional Chinese medicine ICH are analyzed. The main findings are as follows: (1) The spatial distribution of traditional Chinese medicine ICH practices is cohesive as a whole, with a tendency to aggregate and strong imbalance in distribution. (2) There are significant differences in the classification and number of traditional Chinese medicine ICH. (3) Traditional Chinese medicine ICH practices have obvious differentiation in time and space distribution when factoring in their announcement in batches. (4) Traditional medicine practices have significant spatial autocorrelation. Appreciation of the aforementioned features allows us to gain consensus on the development and evolution of traditional Chinese medicine ICH, and hence can justify how we allocate funds to promote traditional Chinese medicine ICH projects, paying particular attention to researching the historical context, and preserving the extraction of the local characteristics of traditional Chinese medicine ICH

Vibration wave downhole communication technique

To overcome the disadvantages of traditional downhole communication methods, a vibration wave downhole communication technique is proposed, and a vibration wave downhole communication system is developed. This technique has been verified by field test and is applied to separated layer water injection. It is shown by theoretical and test research that transmission of the vibration wave through tubing and casing appears as the alternate distribution of pass-band and stop-band. According to that, a multi-baseband transmission strategy is formulated. The on-off keying modulation and Manchester encoding scheme are used to load the control information into the vibration wave. A generation system of vibration signals is developed to realize the controllable conversion from electric energy into vibration wave energy. A receiving and decoding system of vibration waves, which uses a micro-vibration acceleration sensor as the signal pickup element, is developed too. A test system for vibration wave downhole remote transmission is designed and applied to field test. The feasibility of the technique and the accuracy and reliability of communication system are verified and the attenuation characteristics of casing vibration wave signals are obtained. This technique has been applied to separated layer water injection successfully with wide application prospect in wellbore control field. Key words: vibration wave, downhole communication, on-off-keying modulation, Manchester encoding, magne-tostrictive material, micro-vibration acceleration senso

An Adaptive MAC Protocol Based on IEEE802.15.6 for Wireless Body Area Networks

The application carrier of wireless body area network (WBAN) is human; due to changes in people’s sports status or physical health and other reasons, the business traffic fluctuates greatly, which requires the network to have good adaptability. In addition, the energy consumption problem is also a key factor restricting the applications of the WBAN. At present, the proposed MAC protocol is not highly adaptive and has low energy efficiency. To solve this problem, this paper proposes an adaptive MAC protocol based on IEEE802.15.6 for WBAN (A-MAC). The protocol sets the data to three priorities according to the type of service; the superframe structure of IEEE802.15.6 is improved and reorganized into four phases: the beacon phase, the contention access phase, the noncontention access phase, and the inactive phase. The length of the contention access phase and the noncontention access phase is adjusted according to the proportion of nodes that generate each priority data. The contention access phase is further divided into three subphases, and the length of the subphase is dynamically adjusted according to the data priority. In the contention access phase, all nodes compete for access channel according to the channel access policy. The random data that competes successfully transmits data directly, and the periodic data that competes successfully transmits data in the allocated time slots of the noncontention access phase. Finally through the simulation of the proposed A-MAC protocol and IEEE 802.15.6 MAC protocol and CA-MAC protocol in network performance which were compared, the results show that in terms of throughput, power consumption, and the network time delay, the network performance using A-mac protocol is better than the network performance using IEEE802.15.6 MAC and CA-MAC protocols