Boosting Span-based Joint Entity and Relation Extraction via Squence Tagging Mechanism

Span-based joint extraction simultaneously conducts named entity recognition (NER) and relation extraction (RE) in text span form. Recent studies have shown that token labels can convey crucial task-specific information and enrich token semantics. However, as far as we know, due to completely abstain from sequence tagging mechanism, all prior span-based work fails to use token label in-formation. To solve this problem, we pro-pose Sequence Tagging enhanced Span-based Network (STSN), a span-based joint extrac-tion network that is enhanced by token BIO label information derived from sequence tag-ging based NER. By stacking multiple atten-tion layers in depth, we design a deep neu-ral architecture to build STSN, and each atten-tion layer consists of three basic attention units. The deep neural architecture first learns seman-tic representations for token labels and span-based joint extraction, and then constructs in-formation interactions between them, which also realizes bidirectional information interac-tions between span-based NER and RE. Fur-thermore, we extend the BIO tagging scheme to make STSN can extract overlapping en-tity. Experiments on three benchmark datasets show that our model consistently outperforms previous optimal models by a large margin, creating new state-of-the-art results.Comment: 10pages, 6 figures, 4 table

Win-Win Cooperation: Bundling Sequence and Span Models for Named Entity Recognition

For Named Entity Recognition (NER), sequence labeling-based and span-based paradigms are quite different. Previous research has demonstrated that the two paradigms have clear complementary advantages, but few models have attempted to leverage these advantages in a single NER model as far as we know. In our previous work, we proposed a paradigm known as Bundling Learning (BL) to address the above problem. The BL paradigm bundles the two NER paradigms, enabling NER models to jointly tune their parameters by weighted summing each paradigm's training loss. However, three critical issues remain unresolved: When does BL work? Why does BL work? Can BL enhance the existing state-of-the-art (SOTA) NER models? To address the first two issues, we implement three NER models, involving a sequence labeling-based model--SeqNER, a span-based NER model--SpanNER, and BL-NER that bundles SeqNER and SpanNER together. We draw two conclusions regarding the two issues based on the experimental results on eleven NER datasets from five domains. We then apply BL to five existing SOTA NER models to investigate the third issue, consisting of three sequence labeling-based models and two span-based models. Experimental results indicate that BL consistently enhances their performance, suggesting that it is possible to construct a new SOTA NER system by incorporating BL into the current SOTA system. Moreover, we find that BL reduces both entity boundary and type prediction errors. In addition, we compare two commonly used labeling tagging methods as well as three types of span semantic representations

C1q/TNF-related protein 3 (CTRP3) and 9 (CTRP9) concentrations are decreased in patients with heart failure and are associated with increased morbidity and mortality.

BACKGROUND: Biochemical marker has revolutionized the approach to the diagnosis of heart failure. However, it remains difficult to assess stability of the patient. As such, novel means of stratifying disease severity are needed. C1q/TNF-Related Protein 3 (CTRP3) and C1q/TNF-Related Protein 9 (CTRP9) are novel adipokines that contribute to energy homeostasis with additional anti-inflammatory and anti-ischemic properties. The aim of our study is to evaluate concentrations of CTRP3 and CTRP9 in patients with HFrEF (heart failure with reduced ejection fraction) and whether associated with mortality. METHODS: Clinical data and plasma were obtained from 176 healthy controls and 168 patients with HFrEF. CTRP3 and CTRP9 levels were evaluated by enzyme-linked immunosorbent assay. RESULTS: Both CTRP3 and CTRP9 concentrations were significantly decreased in the HFrEF group compared to the control group (p \u3c 0.001). Moreover, patients with higher New York Heart Association class had significantly lower CTRP3 or CTRP9 concentrations. Correlation analysis revealed that CTRP3 and CTRP9 levels were positively related with LVEF% (CTRP3, r = 0.556, p \u3c 0.001; CTRP9, r = 0.526, p \u3c 0.001) and negatively related with NT-proBNP levels (CTRP3, r = - 0.454, p \u3c 0.001; CTRP9, r = - 0.483, p \u3c 0.001). After a follow up for 36 months, after adjusted for age, LVEF and NT-proBNP, we observed that CTRP3 or CTRP9 levels below the 25th percentile was a predictor of total mortality (CTRP3,HR:1.93,95%CI1.03~3.62,P = 0.042;CTRP9,HR:1.98,95%CI:1.02~3.85,P = 0.044) and hospitalizations (CTRP3,HR:2.34,95% CI:1.43~3.82,P = 0.001;CTRP9,HR:2.67,95%CI:1.58~4.50,P \u3c 0.001). CONCLUSIONS: CTRP3 and CTRP9 are decreased in patients with HFrEF, proportionate to disease severity, and each is associated with increased morbidity and mortality. TRIAL REGISTRATION: NCT01372800 . Registered May 2011

Priprema, identifikacija i antioksidacijska svojstva kelatnog kompleksa željeza i oligopeptida izoliranog iz mesa japanske svilaste crne kokoši (Gallus galllus domesticus Brisson)

Black-bone silky fowl iron(II)-oligopeptide chelate was synthesized from iron(II) solution and the black-bone silky fowl oligopeptide, which was extracted from the muscle protein of black-bone silky fowl (Gallus gallus domesticus Brisson). Orthogonal array analysis was used to determine the optimal conditions for the iron(II)-oligopeptide chelate preparation. Ultraviolet-visible (UV-Vis) spectroscopy, electron microscopy, and Fourier transform infrared (FTIR) spectroscopy were used to identify the structure of iron(II)-oligopeptide chelate. 2-Diphenyl-1-picrylhydrazyl (DPPH) and superoxide radical scavenging assays were performed to compare the antioxidant abilities of the black-bone silky fowl oligopeptide and iron(II)-oligopeptide chelate. The optimal conditions for iron(II) oligopeptide chelate preparation were 4 % of the black-bone silky fowl oligopeptide and a ratio of the black-bone silky fowl oligopeptide to FeCl2·4H2O of 5:1 at pH=4. Under these conditions, the chelation rate was (84.9±0.2) % (p<0.05), and the chelation yield was (40.3±0.1) % (p<0.05). The structures detected with UV-Vis spectroscopy, electron microscopy and FTIR spectra changed significantly after chelation, suggesting that Fe(II) ions formed coordinate bonds with carboxylate (-RCOO¯) and amino (-NH2) groups in the oligopeptides, confirming that this is a new oligopeptide-iron chelate. The iron(II)-oligopeptide chelate had stronger scavenging activity towards DPPH and superoxide radicals than did the black-bone silky fowl oligopeptide.Kelatni kompleks željeza i oligopeptida sintetiziran je dodatkom praha proteina izoliranog iz mesa japanske svilaste crne kokoši (Gallus galllus domesticus Brisson) otopini iona Fe2+. Optimalni uvjeti keliranja određeni su pomoću ortogonalnog plana. Struktura kelata ispitana je pomoću UV-Vis spektroskopije, elektronskog mikroskopa i FTIR spektroskopije. Uspoređena je antioksidacijska aktivnost oligopeptida i kelata, i to ispitivanjem sposobnosti uklanjanja DPPH i superoksidnih radikala. Optimalni uvjeti keliranja bili su: omjer mase oligopeptida i volumena otopine od 4 %, maseni omjer oligopeptida i otopine željezovog(II) klorida od 5:1 i pH-vrijednost od 4. Pri tim je uvjetima uspješnost keliranja bila (84,9±0,2) % (p˂0,05), a prinos kelata (40,3±0,1) % (p˂0,05). Isptivanjem spojeva pomoću UV-Vis spektroskopije, elektronskog mikroskopa i FTIR spektroskopije utvrđeno je da se struktura kelata bitno promijenila, te da je nastao novi spoj, najvjerojatnije vezivanjem iona Fe2+ s karboksilnom i amino skupinom oligopeptida. Kelatni kompleks imao je izraženiju sposobnost uklanjanja DPPH i superoksidnih radikala od oligopeptida

PMET: Precise Model Editing in a Transformer

Model editing techniques modify a minor proportion of knowledge in Large Language Models (LLMs) at a relatively low cost, which have demonstrated notable success. Existing methods assume Transformer Layer (TL) hidden states are values of key-value memories of the Feed-Forward Network (FFN). They usually optimize the TL hidden states to memorize target knowledge and use it to update the weights of the FFN in LLMs. However, the information flow of TL hidden states comes from three parts: Multi-Head Self-Attention (MHSA), FFN, and residual connections. Existing methods neglect the fact that the TL hidden states contains information not specifically required for FFN. Consequently, the performance of model editing decreases. To achieve more precise model editing, we analyze hidden states of MHSA and FFN, finding that MHSA encodes certain general knowledge extraction patterns. This implies that MHSA weights do not require updating when new knowledge is introduced. Based on above findings, we introduce PMET, which simultaneously optimizes Transformer Component (TC, namely MHSA and FFN) hidden states, while only using the optimized TC hidden states of FFN to precisely update FFN weights. Our experiments demonstrate that PMET exhibits state-of-the-art performance on both the COUNTERFACT and zsRE datasets. Our ablation experiments substantiate the effectiveness of our enhancements, further reinforcing the finding that the MHSA encodes certain general knowledge extraction patterns and indicating its storage of a small amount of factual knowledge. Our code is available at https://github.com/xpq-tech/PMET.git.Comment: Preprint. Under revie

Sluggish and Chemically-Biased Interstitial Diffusion in Concentrated Solid Solution Alloys: Mechanisms and Methods

Interstitial diffusion is a pivotal process that governs the phase stability and irradiation response of materials in non-equilibrium conditions. In this work, we study sluggish and chemically-biased interstitial diffusion in Fe-Ni concentrated solid solution alloys (CSAs) by combining machine learning (ML) and kinetic Monte Carlo (kMC), where ML is used to accurately and efficiently predict the migration energy barriers on-the-fly. The ML-kMC reproduces the diffusivity that was reported by molecular dynamics results at high temperatures. With this powerful tool, we find that the observed sluggish diffusion and the "Ni-Ni-Ni"-biased diffusion in Fe-Ni alloys are ascribed to a unique "Barrier Lock" mechanism, whereas the "Fe-Fe-Fe"-biased diffusion is influenced by a "Component Dominance" mechanism. Inspired by the mentioned mechanisms, a practical AvgS-kMC method is proposed for conveniently and swiftly determining interstitial-mediated diffusivity by only relying on the mean energy barriers of migration patterns. Combining the AvgS-kMC with the differential evolutionary algorithm, an inverse design strategy for optimizing sluggish diffusion properties is applied to emphasize the crucial role of favorable migration patterns.Comment: 30 pages,9 figure

Partial Covering Arrays: Algorithms and Asymptotics

A covering array $\mathsf{CA}(N;t,k,v)$ is an $N\times k$ array with entries in $\{1, 2, \ldots , v\}$, for which every $N\times t$ subarray contains each $t$-tuple of $\{1, 2, \ldots , v\}^t$ among its rows. Covering arrays find application in interaction testing, including software and hardware testing, advanced materials development, and biological systems. A central question is to determine or bound $\mathsf{CAN}(t,k,v)$, the minimum number $N$ of rows of a $\mathsf{CA}(N;t,k,v)$. The well known bound $\mathsf{CAN}(t,k,v)=O((t-1)v^t\log k)$ is not too far from being asymptotically optimal. Sensible relaxations of the covering requirement arise when (1) the set $\{1, 2, \ldots , v\}^t$ need only be contained among the rows of at least $(1-\epsilon)\binom{k}{t}$ of the $N\times t$ subarrays and (2) the rows of every $N\times t$ subarray need only contain a (large) subset of $\{1, 2, \ldots , v\}^t$. In this paper, using probabilistic methods, significant improvements on the covering array upper bound are established for both relaxations, and for the conjunction of the two. In each case, a randomized algorithm constructs such arrays in expected polynomial time