Boosting Inference Efficiency: Unleashing the Power of Parameter-Shared Pre-trained Language Models

Parameter-shared pre-trained language models (PLMs) have emerged as a successful approach in resource-constrained environments, enabling substantial reductions in model storage and memory costs without significant performance compromise. However, it is important to note that parameter sharing does not alleviate computational burdens associated with inference, thus impeding its practicality in situations characterized by limited stringent latency requirements or computational resources. Building upon neural ordinary differential equations (ODEs), we introduce a straightforward technique to enhance the inference efficiency of parameter-shared PLMs. Additionally, we propose a simple pre-training technique that leads to fully or partially shared models capable of achieving even greater inference acceleration. The experimental results demonstrate the effectiveness of our methods on both autoregressive and autoencoding PLMs, providing novel insights into more efficient utilization of parameter-shared models in resource-constrained settings.Comment: EMNLP 2023 Finding

Expression and characterization of the UL31 protein from duck enteritis virus

<p>Abstract</p> <p>Background</p> <p>Previous studies indicate that the UL31 protein and its homology play similar roles in nuclear egress of all <it>herpesviruses</it>. However, there is no report on the UL31 gene product of DEV. In this study, we expressed and presented the basic properties of the DEV UL31 product.</p> <p>Results</p> <p>The entire ORF of the UL31 was cloned into pET 32a (+) prokaryotic expression vector. <it>Escherichia coli </it>BL21(DE3) competent cells were transformed with the construct followed by the induction of protein expression by the addition of IPTG. Band corresponding to the predicted sizes (55 kDa) was produced on the SDS-PAGE. Over expressed 6×His-UL31 fusion protein was purified by nickel affinity chromatography. The DEV UL31 gene product has been identified by using a rabbit polyclonal antiserum raised against the purified protein. A protein of approximate 35 kDa that reacted with the antiserum was detected in immunoblots of DEV-infected cellular lysates, suggesting that the 35 kDa protein was the primary translation product of the UL31 gene. RT-PCR analyses revealed that the UL31 gene was transcribed most abundantly during the late phase of replication. Subsequently, Immunofluorescence analysis revealed that the protein was widespread speckled structures in the nuclei of infected cells. Western blotting of purified virion preparations showed that UL31 was a component of intracellular virions but was absent from mature extracellular virions. Finally, an Immunofluorescence assay was established to study the distribution of the UL31 antigen in tissues of artificially DEV infected ducks. The results showed that the UL31 antigen was primarily located in the cells of digestive organs and immunological organs.</p> <p>Conclusion</p> <p>In this work, we present the basic properties of the DEV UL31 product. The results indicate that DEV UL31 shares many similarities with its HSV or PRV homolog UL31 and suggest that functional cross-complementation is possible between members of the <it>Alpha</it>herpesvirus subfamily. Furthermore, in vivo experiments with ducks infected with UL31-defective isolates of DEV will also be of importance in order to assess the possible role of the UL31 protein in viral pathogenesis. These properties of the UL31 protein provide a prerequisite for further functional analysis of this gene.</p

TSUP Speaker Diarization System for Conversational Short-phrase Speaker Diarization Challenge

This paper describes the TSUP team's submission to the ISCSLP 2022 conversational short-phrase speaker diarization (CSSD) challenge which particularly focuses on short-phrase conversations with a new evaluation metric called conversational diarization error rate (CDER). In this challenge, we explore three kinds of typical speaker diarization systems, which are spectral clustering(SC) based diarization, target-speaker voice activity detection(TS-VAD) and end-to-end neural diarization(EEND) respectively. Our major findings are summarized as follows. First, the SC approach is more favored over the other two approaches under the new CDER metric. Second, tuning on hyperparameters is essential to CDER for all three types of speaker diarization systems. Specifically, CDER becomes smaller when the length of sub-segments setting longer. Finally, multi-system fusion through DOVER-LAP will worsen the CDER metric on the challenge data. Our submitted SC system eventually ranks the third place in the challenge

Grounded source transient electromagnetic 3D forward modeling with the spectral-element method and its application in hydraulic fracturing monitoring

A long wire with large current source transient electromagnetic (TEM) monitoring, with a large detection depth, low cost, safety, and environmental protection, has unique advantages in the testing and identification of unconventional reservoir fluid and the evaluation of stimulated reservoir volume. So, the TEM 3D forward modeling method has become a research hotspot. Although the finite-element method (FEM) is a type of numerical algorithm that has been widely applied in three-dimensional (3D) electromagnetic field forward modeling, the efficiency and accuracy of FEM require further improvement in order to meet the demand of fast 3D inversion. By increasing the order of the basis function and adjusting the principle of mesh discretization, the precision of the mixed-order spectral-element (SEM) result will be increased. The backward Euler scheme is an unconditionally stable technique which can ignore the impact of the scale of the time step. To achieve a better description of the nonlinear electromagnetic (EM) response of the grounded source TEM method and to optimize the efficiency and accuracy/precision of the 3D TEM forward modeling method significantly, we proposed the use of 3D TEM forward modeling based on the mixed-order SEM and the backward Euler scheme, which can obtain more accurate EM results with fewer degrees of freedom. To check its accuracy and efficiency, the 1D and 3D layered models are applied to compare the SEM results with the semi-analytical and FEM solutions. In addition, we analyzed the accuracy and efficiency of the SEM method for different types of order basis functions. Finally, we calculated the long-wire source TEM response for a practical 3D earth model of a shale gas reservoir for fracturing monitoring and tested the feasibility of the TEM method in a hydraulic fracturing monitoring area to further demonstrate the flexibility of the SEM method

Fire-Needle Moxibustion for the Treatment of Knee Osteoarthritis: A Meta-Analysis

Objectives. The aim of this study was to evaluate the effectiveness of fire-needle moxibustion as an intervention in the treatment of knee osteoarthritis (KOA). Methods. An updated meta-analysis of randomized controlled trials (RCTs) on fire-needle moxibustion in treating KOA was conducted by searching PubMed, Embase, the Cochrane Library, Web of Science, Wanfang database, and the Chinese Medical Database (CNKI) since their inception through March 2016. The meta-analysis was performed using RevMan 5.3. Results. Thirteen RCTs were identified in the systematic study which consisted of 1179 participants. Fire-needle moxibustion treatment group had a statistical significance on recovery rate as well as recovery and marked-improvement rate compared with control group. Subgroup analysis indicated that there was significant difference between fire-needle moxibustion group and control group. However, GRADE analysis indicated that the quality of evidence for all outcomes was relatively low. Only two of 13 studies reported adverse reactions (difficulty in movement and intolerance of cold). Conclusion. This meta-analysis suggests that fire-needle moxibustion is more effective than control group in symptom management of KOA. Further high quality trials should be conducted to evaluate the effectiveness of fire-needle moxibustion on KOA

Rotational Symmetry Breaking in Superconducting Nickelate Nd0.8Sr0.2NiO2 Films

The infinite-layer nickelates, isostructural to the high-Tc cuprate superconductors, have emerged as a promising platform to host unconventional superconductivity and stimulated growing interest in the condensed matter community. Despite considerable attention, the superconducting pairing symmetry of the nickelate superconductors, the fundamental characteristic of a superconducting state, is still under debate. Moreover, the strong electronic correlation in the nickelates may give rise to a rich phase diagram, where the underlying interplay between the superconductivity and other emerging quantum states with broken symmetry is awaiting exploration. Here, we study the angular dependence of the transport properties of the infinite-layer nickelate Nd0.8Sr0.2NiO2 superconducting films with Corbino-disk configuration. The azimuthal angular dependence of the magnetoresistance (R(φ)) manifests the rotational symmetry breaking from isotropy to four-fold (C4) anisotropy with increasing magnetic field, revealing a symmetry-breaking phase transition. Approaching the low-temperature and large-magnetic-field regime, an additional two-fold (C2) symmetric component in the R(φ) curves and an anomalous upturn of the temperature-dependent critical field are observed simultaneously, suggesting the emergence of an exotic electronic phase. Our work uncovers the evolution of the quantum states with different rotational symmetries in nickelate superconductors and provides deep insight into their global phase diagram

Low-dose Aspirin prevents hypertension and cardiac fibrosis when thromboxane A2 is unrestrained.

Abstract Enhanced platelet activation has been reported in patients with essential hypertension and heart failure. The possible contribution of platelet-derived thromboxane (TX)A2 in their pathophysiology remains unclear. We investigated the systemic TXA2 biosynthesis in vivo and gene expression of its receptor TP in 22 essential hypertension patients and a mouse model of salt-sensitive hypertension. The contribution of platelet TXA2 biosynthesis on enhanced blood pressure (BP) and overload-induced cardiac fibrosis was explored in mice by treating with low-dose Aspirin, resulting in selective inhibition of platelet cyclooxygenase (COX)-1-dependent TXA2 generation. In essential hypertensive patients, systemic biosynthesis of TXA2 [assessed by measuring its urinary metabolites (TXM) reflecting predominant platelet source] was enhanced together with higher gene expression of circulating leukocyte TP and TGF-β, vs. normotensive controls. Similarly, in hypertensive mice with prostacyclin (PGI2) receptor (IP) deletion (IPKO) fed with a high-salt diet, enhanced urinary TXM, and left ventricular TP overexpression were detected vs. normotensive wildtype (WT) mice. Increased cardiac collagen deposition and profibrotic gene expression (including TGF-β) was found. Low-dose Aspirin administration caused a selective inhibition of platelet TXA2 biosynthesis and mitigated enhanced blood pressure, cardiac fibrosis, and left ventricular profibrotic gene expression in IPKO but not WT mice. Moreover, the number of myofibroblasts and extravasated platelets in the heart was reduced. In cocultures of human platelets and myofibroblasts, platelet TXA2 induced profibrotic gene expression, including TGF-β1. In conclusion, our results support tailoring low-dose Aspirin treatment in hypertensive patients with unconstrained TXA2/TP pathway to reduce blood pressure and prevent early cardiac fibrosis

Comprehensive comparative analysis of prognostic value of serum systemic inflammation biomarkers for colorectal cancer: Results from a large multicenter collaboration

BackgroundThe incidence of colorectal cancer (CRC) is common and reliable biomarkers are lacking. We aimed to systematically and comprehensively compare the ability of various combinations of serum inflammatory signatures to predict the prognosis of CRC. Moreover, particular attention has been paid to the clinical feasibility of the newly developed inflammatory burden index (IBI) as a prognostic biomarker for CRC.MethodsThe discrimination capacity of the biomarkers was compared using receiver operating characteristic curves and Harrell’s C-index. Kaplan-Meier curves and log-rank tests were used to compare survival differences between the groups. Cox proportional hazard regression analysis was used to determine the independent prognostic factors. Logistic regression analysis was used to assess the relationship between IBI, short-term outcomes, and malnutrition.ResultsIBI had the optimal prediction accuracy among the systemic inflammation biomarkers for predicting the prognosis of CRC. Taking IBI as a reference, none of the remaining systemic inflammation biomarkers showed a gain. Patients with high IBI had significantly worse overall survival than those with low IBI (56.7% vs. 80.2%; log-rank P&lt;0.001). Multivariate Cox regression analysis showed that continuous IBI was an independent risk factor for the prognosis of CRC patients (hazard ratio = 1.165, 95% confidence interval [CI] = 1.043–1.302, P&lt;0.001). High IBI was an independent risk factor for short-term outcomes (odds ratio [OR] = 1.537, 95% CI = 1.258–1.878, P&lt;0.001), malnutrition (OR = 2.996, 95% CI = 1.471–6.103, P=0.003), and recurrence (OR = 1.744, 95% CI = 1.176–2.587, p = 0.006) in CRC patients.ConclusionsIBI, as a reflection of systemic inflammation, is a feasible and promising biomarker for assessing the prognosis of CRC patients

DHAV-1 Inhibits Type I Interferon Signaling to Assist Viral Adaption by Increasing the Expression of SOCS3

Duck hepatitis A virus type 1 (DHAV-1) is one of the most lethal pathogens in the duck industry. The attenuated vaccine (the CH60 strain) is cultivated through serial passage in chicken embryos and is widely used for the prevention and control of the disease. However, the specific mechanism underlying its adaptation in chicken embryos has not been fully elucidated. In this study, we first infected chicken embryo fibroblasts (CEFs) with the DHAV-1 CH60 strain. The peak of viral proliferation occurred within 36–48 h post-infection. The different DHAV-1 strains significantly induced the expression of IFNα, IFNγ, and Suppressor of cytokine signaling 3 (SOCS3) in CEFs, and we found that SOCS3 overexpression significantly promoted viral replication. Furthermore, SOCS3 overexpression significantly inhibited the expression of IFNα but promoted the expression of IFNγ. In addition, SOCS3 overexpression clearly decreased the mRNA levels of STAT1 and STAT3 in the Janus kinase (JAK)-STAT signaling pathway and inhibited the expression of the antiviral proteins MX1 and OASL. Immune-precipitation assays indicated that SOCS3 and IFNα do not physically interact. Subcellular localization of SOCS3 and IFNα revealed that SOCS3 was mainly located in the nucleus and cytoplasm, while IFNα was located only in the cytoplasm. Co-localization of these two proteins was not observed in the cytoplasm. In conclusion, the DHAV-1 CH60 strain may inhibit the expression of IFNα by increasing the SOCS3 protein and SOCS3 can in turn decrease STAT1 and STAT3 mRNA levels, thereby inhibiting the antiviral protein MX1 and ultimately promoting viral proliferation, indirectly assisting in viral adaptation in chicken embryos