A Quantitative Review on Language Model Efficiency Research

Language models (LMs) are being scaled and becoming powerful. Improving their efficiency is one of the core research topics in neural information processing systems. Tay et al. (2022) provided a comprehensive overview of efficient Transformers that have become an indispensable staple in the field of NLP. However, in the section of "On Evaluation", they left an open question "which fundamental efficient Transformer one should consider," answered by "still a mystery" because "many research papers select their own benchmarks." Unfortunately, there was not quantitative analysis about the performances of Transformers on any benchmarks. Moreover, state space models (SSMs) have demonstrated their abilities of modeling long-range sequences with non-attention mechanisms, which were not discussed in the prior review. This article makes a meta analysis on the results from a set of papers on efficient Transformers as well as those on SSMs. It provides a quantitative review on LM efficiency research and gives suggestions for future research.Comment: 29 pages, 24 table

Embedding Mental Health Discourse for Community Recommendation

Our paper investigates the use of discourse embedding techniques to develop a community recommendation system that focuses on mental health support groups on social media. Social media platforms provide a means for users to anonymously connect with communities that cater to their specific interests. However, with the vast number of online communities available, users may face difficulties in identifying relevant groups to address their mental health concerns. To address this challenge, we explore the integration of discourse information from various subreddit communities using embedding techniques to develop an effective recommendation system. Our approach involves the use of content-based and collaborative filtering techniques to enhance the performance of the recommendation system. Our findings indicate that the proposed approach outperforms the use of each technique separately and provides interpretability in the recommendation process.Comment: Accepted to the 4th workshop on Computational Approaches to Discourse (CODI-2023) at ACL 202

Complete genome assembly and characterization of an outbreak strain of the causative agent of swine erysipelas – Erysipelothrix rhusiopathiae SY1027

BACKGROUND: Erysipelothrix rhusiopathiae is the causative agent of animal erysipelas and, to a fewer occurrences, human erysipeloid. It is ubiquitous in nature and commensal in diverse species of animals, wild or domestic, from mammals and birds to reptiles and fish. Mechanisms of its virulence and pathogenicity are poorly understood. RESULTS: Making use of the complete genome sequencing of E. rhusiopathiae strain SY1027 and comparative genome analysis between the three highly pathogenic strains (SY1027, Fujisawa and ATCC19414), the genomic structure and putative functional elements, such as pathogenicity island (PAI)-like regions, potential virulence factors and horizontal transferring genes of the bacteria are identified. Strain SY1027 genome is 1,752,910 base pairs long, just 30 kilobases smaller than strain Fujisawa, with the same GC level of 36.36%. It contains 1,845 open reading frames (ORF) predicted by GLIMMER 3.02, of which 1,775 were annotated by PGAAP, 1,757 (~95.23%) were annotated by NCBI nr blast, 1,209 by COG database and 1,076 by KEGG database. 37 potential virulence factors were annotated in strain SY1027 by VFDB, while 19 (~51.35%) of them are common in the 2 strains, 7 of which are potentially related to antibiotic resistance and highly conserved (~98-100% match identity (ID)) amongst the three strains of E. rhusiopathiae and modestly homologous to other gastrointestinal tract-inhabiting Firmicutes (~40% match ID), e.g. Clostridium spp., Enterococcus spp. Genomic island- and pathogenicity island-like regions were also predicted, in which some showed association with tRNA and potential virulence factors. CONCLUSION: Complete genome sequencing of Erysipelothrix rhusiopathiae, the causative agent of animal erysipelas, was performed. Molecular identification of various genomic elements pave the way to the better understanding of mechanisms underlying metabolic capabilities, pathogenicity of swine erysipelas and prospective vaccine targets besides the widely used SpaA antigens

Multimodal Graph Learning for Modeling Emerging Pandemics with Big Data

Accurate forecasting and analysis of emerging pandemics play a crucial role in effective public health management and decision-making. Traditional approaches primarily rely on epidemiological data, overlooking other valuable sources of information that could act as sensors or indicators of pandemic patterns. In this paper, we propose a novel framework called MGL4MEP that integrates temporal graph neural networks and multi-modal data for learning and forecasting. We incorporate big data sources, including social media content, by utilizing specific pre-trained language models and discovering the underlying graph structure among users. This integration provides rich indicators of pandemic dynamics through learning with temporal graph neural networks. Extensive experiments demonstrate the effectiveness of our framework in pandemic forecasting and analysis, outperforming baseline methods across different areas, pandemic situations, and prediction horizons. The fusion of temporal graph learning and multi-modal data enables a comprehensive understanding of the pandemic landscape with less time lag, cheap cost, and more potential information indicators

Formation and migration of vacancy defects in GeSe and SnSe

The GeSe and SnSe have great potential in nuclear detector devices. Under irradiation, the formation and migration of point defects may affect their properties and performance significantly. In this study, a comparative study of vacancy formation and migration in GeSe and SnSe has been carried out by a first-principles method. It is shown that in both compounds the cation vacancies are generally much easier to form than anion vacancies, and the cation vacancies are generally easier to migrate than anion vacancies. For both Ge vacancy and Sn vacancy, the migration is anisotropic and the [322] direction is the most favorable migration pathway. The migration energy barrier are 0.54 eV for Ge vacancy and 0.46–0.52 eV for Sn vacancy, suggesting that vacancy clusters are relatively easy to form in both compounds, which may influence the application of GeSe and SnSe in nuclear detector devices

Highly Selective Conversion of CH4 to High Value-Added C1 Oxygenates over Pd Loaded ZnTi-LDH

The selective oxidation of methane to high value-added liquid oxygenated compounds under mild conditions is of great significance to promote the efficient utilization of the carbon source, but it also faces the dilemma of low activity and over-oxidation. Here, ZnTi-layered double hydroxides (LDH)-A200 photocatalysts with Pd loading are prepared to achieve efficient oxidation of methane, with O2 as an oxidant under ambient condition. The highest generation rate of C1 liquid products (methanol and formaldehyde) reaches 4924.47 µmol g−1 h−1 with a selectivity close to 100% over 0.5Pd-ZnTi-LDH-A200, which is 20 times higher than that of bare ZnTi-LDH-A200. The photochemical results show that the modified photocatalysts present much higher generation and separation efficiency of electron-hole pairs. In situ X–ray photoelectron spectroscopy indicates that Pd nanoparticles are the hole acceptor, which is beneficial to charge separation in the photocatalysis. Furthermore, electron spinresonance spectroscopy and temperature–programmed–desorption analysis prove that Pd loading is helpful to the adsorption of methane and oxygen on the surface of ZnTi-LDH-A200, promoting the production of reactive oxygen species and activation of methane. All these factors work together to promote the efficient conversion of CH4 to high value-added C1 oxygenates

Improvement of the Photoelectrochemical Stability of Cu2O Photocathode by Ph—CΞC—Cu Grafting

As one of the most efficient photocathodes, Cu2O has attracted substantial attention because of its theoretically high solar-to-hydrogen (STH) efficiency. However, its applications in photoelectrochemical (PEC) fields are severely restricted by the poor stability derived from serious photocorrosion. In this work, high-quality phenylethynyl copper (Ph—CΞC—Cu) are successfully self-assembled on the surface of Cu2O photocathode by a novel photothermal method to improve its photostability. With the protection of the Ph—CΞC—Cu layer, 85% of the initial photocurrent density can be remained, while only 28% of initial photocurrent density is left on bare Cu2O photocathode prepared on a copper foam (CF) substrate. The significantly improved photostability of Cu2O photocathode by the Ph—CΞC—Cu protective layer is attributed to its strong hydrophobicity, which can efficiently inhibit the corrosion of Cu2O by the aqueous electrolyte solution due to its special crystal structure. Based on the obtained Ph—CΞC—Cu/Cu2O photocathode, a two-photoelectrode cell with excellent stability (>5 h) has also been successfully constructed for water splitting without the need of an electric bias

Comparison of serum anti-Mullerian hormone levels following hysterectomy and myomectomy for benign gynaecological conditions.

OBJECTIVE: To compare serum anti-Mullerian hormone (AMH) levels following hysterectomy and myomectomy. STUDY DESIGN: Prospective longitudinal observational study. Serum AMH, follicle stimulating hormone (FSH) and luteinizing hormone (LH) levels were measured pre-operatively (T1) and 2 days (T2) and 3 months (T3) following hysterectomy and myomectomy in 70 women aged 36-45 years. Hysterectomy (laparoscopy-assisted vaginal hysterectomy=10; total abdominal hysterectomy=25) with conservation of both ovaries for benign diseases of the uterus was performed in 35 women, and myomectomy (laparoscopy myomectomy=15; open myomectomy=20) was performed in another 35 women. The follow-up period was 3 months following surgery. The results were analysed using the t-test or one-way analysis of variance by repeated-measures ANOVA. RESULTS: Serum AMH in the hysterectomy group was 1.08+/-0.77 ng/ml at T1, 0.78+/-0.58 ng/ml at T2 and 0.81+/-0.58 ng/ml at T3; the level was significantly lower at T2 and T3 compared with T1. In the myomectomy group, the corresponding values were 1.54+/-0.95 ng/ml, 1.18+/-0.77 ng/ml and 1.50+/-0.58 ng/ml; serum AMH was significantly lower at T2 compared with T1, but the difference between T3 and T1 was not significant. There were no significant differences in serum FSH and LH in either group between these three time points. CONCLUSION: Serum AMH was significantly lower 2 days and 3 months following hysterectomy compared with the pre-operative level. Following myomectomy, serum AMH was significantly lower than the pre-operative level 2 days following the procedure, but was similar to the pre-operative level 3 months after surgery. Therefore, hysterectomy may have a more lasting adverse effect on ovarian reserve than myomectomy. A long-term study of AMH levels is needed.postprin

Pollen and Phytoliths from Fired Ancient Potsherds as Potential Indicators for Deciphering Past Vegetation and Climate in Turpan, Xinjiang, NW China

It is demonstrated that palynomorphs can occur in fired ancient potsherds when the firing temperature was under 350°C. Pollen and phytoliths recovered from incompletely fired and fully fired potsherds (ca. 2700 yrs BP) from the Yanghai Tombs, Turpan, Xinjiang, NW China can be used as potential indicators for reconstructing past vegetation and corresponding climate in the area. The results show a higher rate of recovery of pollen and phytoliths from incompletely fired potsherds than from fully fired ones. Charred phytoliths recovered from both fully fired and incompletely fired potsherds prove that degree and condition of firing result in a permanent change in phytolith color. The palynological data, together with previous data of macrobotanical remains from the Yanghai Tombs, suggest that temperate vegetation and arid climatic conditions dominated in the area ca. 2700 yrs BP