SoftMCL: Soft Momentum Contrastive Learning for Fine-grained Sentiment-aware Pre-training

The pre-training for language models captures general language understanding but fails to distinguish the affective impact of a particular context to a specific word. Recent works have sought to introduce contrastive learning (CL) for sentiment-aware pre-training in acquiring affective information. Nevertheless, these methods present two significant limitations. First, the compatibility of the GPU memory often limits the number of negative samples, hindering the opportunities to learn good representations. In addition, using only a few sentiment polarities as hard labels, e.g., positive, neutral, and negative, to supervise CL will force all representations to converge to a few points, leading to the issue of latent space collapse. This study proposes a soft momentum contrastive learning (SoftMCL) for fine-grained sentiment-aware pre-training. Instead of hard labels, we introduce valence ratings as soft-label supervision for CL to fine-grained measure the sentiment similarities between samples. The proposed SoftMCL is conducted on both the word- and sentence-level to enhance the model's ability to learn affective information. A momentum queue was introduced to expand the contrastive samples, allowing storing and involving more negatives to overcome the limitations of hardware platforms. Extensive experiments were conducted on four different sentiment-related tasks, which demonstrates the effectiveness of the proposed SoftMCL method. The code and data of the proposed SoftMCL is available at: https://www.github.com/wangjin0818/SoftMCL/.Comment: Accepted by LREC-COLING 202

Germicidal effect of intense pulsed light on Pseudomonas aeruginosa in food processing

BackgroundPseudomonas aeruginosa (P. aeruginosa) can cause serious infections in many parts of the body and is also an underestimated foodborne pathogen. Intense pulsed light sterilization is recognized for its high sterilization efficiency, flexible and safe operation and ease of installation on production lines, which makes up for the shortcomings of several other physical sterilization technologies.MethodsThis experiment studied the killing efficiency of different capacitances (650 μF, 470 μF, and 220 μF) of intense pulsed light on foodborne pathogenic microorganisms P. aeruginosa in the models of liquid food models, 96-well cell plates, and polycarbonate membrane models at room temperature (25°C) and refrigerated (4°C) environments to provide data to support the application of IPL sterilization devices in food processing.ResultsThe IPL was very effective in killing P. aeruginosa in the planktonic state as well as in the early and mature biofilm states, meeting target kill rates of 100%, 99.99%, and 94.33% for a given number of exposures. The biofilms formed in the polycarbonate membrane model and the 96-well plate model were more resistant to killing compared to the planktonic state. To achieve the same bactericidal effect, the number of flashes increased with decreasing capacitance.ConclusionThe bactericidal effect of IPL on P. aeruginosa was significantly influenced by the state of the bacterium. The larger the capacitance the higher the number of pulses and the better the sterilization effect on P. aeruginosa

Understanding the Distillation Process from Deep Generative Models to Tractable Probabilistic Circuits

Probabilistic Circuits (PCs) are a general and unified computational framework for tractable probabilistic models that support efficient computation of various inference tasks (e.g., computing marginal probabilities). Towards enabling such reasoning capabilities in complex real-world tasks, Liu et al. (2022) propose to distill knowledge (through latent variable assignments) from less tractable but more expressive deep generative models. However, it is still unclear what factors make this distillation work well. In this paper, we theoretically and empirically discover that the performance of a PC can exceed that of its teacher model. Therefore, instead of performing distillation from the most expressive deep generative model, we study what properties the teacher model and the PC should have in order to achieve good distillation performance. This leads to a generic algorithmic improvement as well as other data-type-specific ones over the existing latent variable distillation pipeline. Empirically, we outperform SoTA TPMs by a large margin on challenging image modeling benchmarks. In particular, on ImageNet32, PCs achieve 4.06 bits-per-dimension, which is only 0.34 behind variational diffusion models (Kingma et al., 2021)

Personalized LoRA for Human-Centered Text Understanding

Effectively and efficiently adapting a pre-trained language model (PLM) for human-centered text understanding (HCTU) is challenging since user tokens are million-level in most personalized applications and do not have concrete explicit semantics. A standard and parameter-efficient approach (e.g., LoRA) necessitates memorizing numerous suits of adapters for each user. In this work, we introduce a personalized LoRA (PLoRA) with a plug-and-play (PnP) framework for the HCTU task. PLoRA is effective, parameter-efficient, and dynamically deploying in PLMs. Moreover, a personalized dropout and a mutual information maximizing strategies are adopted and hence the proposed PLoRA can be well adapted to few/zero-shot learning scenarios for the cold-start issue. Experiments conducted on four benchmark datasets show that the proposed method outperforms existing methods in full/few/zero-shot learning scenarios for the HCTU task, even though it has fewer trainable parameters. For reproducibility, the code for this paper is available at: https://github.com/yoyo-yun/PLoRA.Comment: Accepted by AAAI 202

Measuring the X-ray luminosities of DESI groups from eROSITA Final Equatorial-Depth Survey: I. X-ray luminosity - halo mass scaling relation

We use the eROSITA Final Equatorial-Depth Survey (eFEDS) to measure the rest-frame 0.1-2.4 keV band X-ray luminosities of $\sim$ 600,000 DESI groups using two different algorithms in the overlap region of the two observations. These groups span a large redshift range of $0.0 \le z_g \le 1.0$ and group mass range of $10^{10.76}h^{-1}M_{\odot} \le M_h \le 10^{15.0}h^{-1}M_{\odot}$. (1) Using the blind detection pipeline of eFEDS, we find that 10932 X-ray emission peaks can be cross matched with our groups, $\sim 38 \%$ of which have signal-to-noise ratio $\rm{S}/\rm{N} \geq 3$ in X-ray detection. Comparing to the numbers reported in previous studies, this matched sample size is a factor of $\sim 6$ larger. (2) By stacking X-ray maps around groups with similar masses and redshifts, we measure the average X-ray luminosity of groups as a function of halo mass in five redshift bins. We find, in a wide halo mass range, the X-ray luminosity, $L_{\rm X}$, is roughly linearly proportional to $M_{h}$, and is quite independent to the redshift of the groups. (3) We use a Poisson distribution to model the X-ray luminosities obtained using two different algorithms and obtain best-fit $L_{\rm X}=10^{28.46\pm0.03}M_{h}^{1.024\pm0.002}$ and $L_{\rm X}=10^{26.73 \pm 0.04}M_{h}^{1.140 \pm 0.003}$ scaling relations, respectively. The best-fit slopes are flatter than the results previously obtained, but closer to a self-similar prediction.Comment: 15 pages, 13 figures, accepted for publication in MNRA

Machine learning prediction of copper ion interference with mercury ion fluorescence signals in food heavy metal detection

Objective: To construct an artificial intelligence prediction model to predict the selectivity of fluorescent probes for Hg2+ in a complex food testing environment in the presence of Cu2+ interference. Methods: Fluorescent probe technology combined with seven advanced classical machine learning models was used to predict and analyze the selectivity of the probe for Hg2+ in the presence of Cu2+ interference, and to compare the prediction effect of each model and select the optimal model. Results: Efficient models with accuracies of 0.786 and 0.810 in the cross-validation and test sets were successfully established based on Molecular 2D Descriptors （Mol2D） and extreme gradient boosting algorithms to accurately predict the probe selectivity of Hg2+ under Cu2+ interference. Conclusion: The model is improved for the design of Hg2+ fluorescent molecular probes by selective prediction, which makes the design of Hg2+ fluorescent probes more efficient and reliable

Proteomics and network pharmacology of Ganshu Nuodan capsules in the prevention of alcoholic liver disease

IntroductionGanshu Nuodan is a liver-protecting dietary supplement composed of Ganoderma lucidum (G. lucidum) spore powder, Pueraria montana (Lour.) Merr. (P. montana), Salvia miltiorrhiza Bunge (S. miltiorrhiza) and Astragalus membranaceus (Fisch.) Bunge. (A. membranaceus). However, its pharmacodynamic material basis and mechanism of action remain unknown.MethodsA mouse model of acute alcohol liver disease (ALD) induced by intragastric administration of 50% alcohol was used to evaluate the hepatoprotective effect of Ganshu Nuodan. The chemical constituents of Ganshu Nuodan were comprehensively identified by UPLC-QTOF/MS, and then its pharmacodynamic material basis and potential mechanism of action were explored by proteomics and network pharmacology.ResultsGanshu Nuodan could ameliorate acute ALD, which is mainly manifested in the significant reduction of alanine aminotransferase (ALT) and aspartate aminotransferase (AST) in serum and malondialdehyde (MDA) content in liver and the remarkably increase of glutathione (GSH) content and superoxide dismutase (SOD) activity in liver. Totally 76 chemical constituents were identified from Ganshu Nuodan by UPLC-QTOF/MS, including 21 quinones, 18 flavonoids, 11 organic acids, 7 terpenoids, 5 ketones, 4 sterols, 3 coumarins and 7 others. Three key signaling pathways were identified via proteomics studies, namely Arachidonic acid metabolism, Retinol metabolism, and HIF-1 signaling pathway respectively. Combined with network pharmacology and molecular docking, six key targets were subsequently obtained, including Ephx2, Lta4h, Map2k1, Stat3, Mtor and Dgat1. Finally, these six key targets and their related components were verified by molecular docking, which could explain the material basis of the hepatoprotective effect of Ganshu Nuodan.ConclusionGanshu Nuodan can protect acute alcohol-induced liver injury in mice by inhibiting oxidative stress, lipid accumulation and apoptosis. Our study provides a scientific basis for the hepatoprotective effect of Ganshu Nuodan in acute ALD mice and supports its traditional application

SkyMath: Technical Report

Large language models (LLMs) have shown great potential to solve varieties of natural language processing (NLP) tasks, including mathematical reasoning. In this work, we present SkyMath, a large language model for mathematics with 13 billion parameters. By applying self-compare fine-tuning, we have enhanced mathematical reasoning abilities of Skywork-13B-Base remarkably. On GSM8K, SkyMath outperforms all known open-source models of similar size and has established a new SOTA performance

Skywork: A More Open Bilingual Foundation Model

In this technical report, we present Skywork-13B, a family of large language models (LLMs) trained on a corpus of over 3.2 trillion tokens drawn from both English and Chinese texts. This bilingual foundation model is the most extensively trained and openly published LLMs of comparable size to date. We introduce a two-stage training methodology using a segmented corpus, targeting general purpose training and then domain-specific enhancement training, respectively. We show that our model not only excels on popular benchmarks, but also achieves \emph{state of the art} performance in Chinese language modeling on diverse domains. Furthermore, we propose a novel leakage detection method, demonstrating that test data contamination is a pressing issue warranting further investigation by the LLM community. To spur future research, we release Skywork-13B along with checkpoints obtained during intermediate stages of the training process. We are also releasing part of our SkyPile corpus, a collection of over 150 billion tokens of web text, which is the largest high quality open Chinese pre-training corpus to date. We hope Skywork-13B and our open corpus will serve as a valuable open-source resource to democratize access to high-quality LLMs

Controlling of structural ordering and rigidity of β-SiAlON:Eu through chemical cosubstitution to approach narrow-band-emission for light-emitting diodes application

The authors are grateful for the financial support of the Ministry of Science and Technology of Taiwan (Contract Nos. MOST 104- 2113-M-002-012-MY3, MOST 104-2119-M-002-027-MY3 and 104-2923-M-002-007-MY3) and Australia Research Council (ARC, FT160100251). The contribution of A. L. was supported by the grant “Preludium” UMO-2014/13/N/ST3/03781 from the National Science Center. The contribution of S. M. was supported by the grant “Iuventus Plus” 0271/IP3/2015/73 from the Ministry of Science and Higher Education. M. G. was supported by Polish National Center for Research and Development with grants no PBS3/A5/48/2015 and PL-TWII/8/2015.Narrow-band green-emitting phosphor β-SiAlON:Eu has been widely used in advanced wide-gamut backlighting de- vices. However, the origins for unusual sharp lines in photoluminescence emission at room temperature and tunable narrow-band- emission tailored by reducing Al-O in β-SiAlON:Eu are still unclear. Here, the presence of sharp-line fine structure in the emission spectra of β-SiAlON:Eu is mainly due to purely electronic transitions (zero phonon lines) and their vibronic repetitions resulted from the multi-microenvironment around Eu2+ ions that has been revealed by relative emission intensity of sharp line depends on excitation wavelength and monotonously increasing decay time. The specific features of the Eu2+ occupying interstitial sites indicate that the effect of crystal field strength can be neglected. Therefore the enhanced rigidity and higher ordering structure of β-SiAlON:Eu with decreasing the substitution of Si–N by Al–O become the main factors in decreasing electron–lattice coupling and reducing inhomo- geneous broadening, favouring the blue-shift and narrow of the emission band, the enhanced thermal stability, as well as the charge state of Eu2+. Our results provide new insights for explaining the reason for narrow-band-emission in β-SiAlON:Eu, which will deliver an impetus for the exploration of phosphors with narrow band and ordering structure.PostprintPeer reviewe