Improving Neural Topic Models with Wasserstein Knowledge Distillation

Topic modeling is a dominant method for exploring document collections on the web and in digital libraries. Recent approaches to topic modeling use pretrained contextualized language models and variational autoencoders. However, large neural topic models have a considerable memory footprint. In this paper, we propose a knowledge distillation framework to compress a contextualized topic model without loss in topic quality. In particular, the proposed distillation objective is to minimize the cross-entropy of the soft labels produced by the teacher and the student models, as well as to minimize the squared 2-Wasserstein distance between the latent distributions learned by the two models. Experiments on two publicly available datasets show that the student trained with knowledge distillation achieves topic coherence much higher than that of the original student model, and even surpasses the teacher while containing far fewer parameters than the teacher's. The distilled model also outperforms several other competitive topic models on topic coherence.Comment: Accepted at ECIR 202

Do Neural Topic Models Really Need Dropout? Analysis of the Effect of Dropout in Topic Modeling

Dropout is a widely used regularization trick to resolve the overfitting issue in large feedforward neural networks trained on a small dataset, which performs poorly on the held-out test subset. Although the effectiveness of this regularization trick has been extensively studied for convolutional neural networks, there is a lack of analysis of it for unsupervised models and in particular, VAE-based neural topic models. In this paper, we have analyzed the consequences of dropout in the encoder as well as in the decoder of the VAE architecture in three widely used neural topic models, namely, contextualized topic model (CTM), ProdLDA, and embedded topic model (ETM) using four publicly available datasets. We characterize the dropout effect on these models in terms of the quality and predictive performance of the generated topics.Comment: Accepted at EACL 202

Improving Contextualized Topic Models with Negative Sampling

Topic modeling has emerged as a dominant method for exploring large document collections. Recent approaches to topic modeling use large contextualized language models and variational autoencoders. In this paper, we propose a negative sampling mechanism for a contextualized topic model to improve the quality of the generated topics. In particular, during model training, we perturb the generated document-topic vector and use a triplet loss to encourage the document reconstructed from the correct document-topic vector to be similar to the input document and dissimilar to the document reconstructed from the perturbed vector. Experiments for different topic counts on three publicly available benchmark datasets show that in most cases, our approach leads to an increase in topic coherence over that of the baselines. Our model also achieves very high topic diversity.Comment: Accepted at 19th International Conference on Natural Language Processing (ICON 2022

Evaluation of genipin-crosslinked chitosan hydrogels as a potential carrier for silver sulfadiazine nanocrystals

This paper was accepted for publication in the journal Colloids and Surfaces B: Biointerfaces and the definitive published version is available at http://dx.doi.org/10.1016/j.colsurfb.2016.06.016In the present study genipin crosslinked chitosan (CHI) hydrogels, which had been constructed and reported in our previous studies (Lei Gao, et al. Colloids Surf. B Biointerfaces. 2014, 117: 398), were further evaluated for their advantage as a carrier for silver sulfadiazine (AgSD) nanocrystal systems. Firstly, AgSD nanocrystals with a mean particle size of 289 nm were prepared by wet milling method and encapsulated into genipin crosslinked CHI hydrogels. AgSD nanocrystals displayed a uniform distribution and very good physical stability in the hydrogel network. Swelling-dependent release pattern was found for AgSD nanocrystals from hydrogels and the release profile could be well fitted with Peppas equation. When AgSD nanocrystals were encapsulated in hydrogels their fibroblast cytotoxicity decreased markedly, and their antibacterial effects against Staphylococcus aureus, Escherichia coli and Pseudomonas aeruginosa were still comparable to unencapsulated AgSD nanocrystals. In vivo evaluation in excision and burn cutaneous wound models in mice showed that AgSD nanocrystal hydrogels markedly decreased the expression of inflammatory cytokine IL-6, but increased the levels of growth factors VEGF-A and TGF-β1. Histopathologically, the wounds treated by hydrogels containing AgSD nanocrystals showed the best healing state compared with commercial AgSD cream, hydrogels containing AgSD bulk powders and blank hydrogels. The wounds treated by AgSD nanocrystal hydrogels were dominated by marked fibroblast proliferation, new blood vessels and thick regenerated epithelial layer. Sirius Red staining assay indicated that AgSD nanocrystal hydrogels resulted in more collagen deposition characterized by a large proportion of type I fibers. Our study suggested that genipin-crosslinked CHI hydrogel was a potential carrier for local antibacterial nanomedicines

Multiplicity dependence of light (anti-)nuclei production in p–Pb collisions at sNN=5.02 TeV

The measurement of the deuteron and anti-deuteron production in the rapidity range −1 < y < 0 as a function of transverse momentum and event multiplicity in p–Pb collisions at √sNN = 5.02 TeV is presented. (Anti-)deuterons are identified via their specific energy loss dE/dx and via their time-of- flight. Their production in p–Pb collisions is compared to pp and Pb–Pb collisions and is discussed within the context of thermal and coalescence models. The ratio of integrated yields of deuterons to protons (d/p) shows a significant increase as a function of the charged-particle multiplicity of the event starting from values similar to those observed in pp collisions at low multiplicities and approaching those observed in Pb–Pb collisions at high multiplicities. The mean transverse particle momenta are extracted from the deuteron spectra and the values are similar to those obtained for p and particles. Thus, deuteron spectra do not follow mass ordering. This behaviour is in contrast to the trend observed for non-composite particles in p–Pb collisions. In addition, the production of the rare 3He and 3He nuclei has been studied. The spectrum corresponding to all non-single diffractive p-Pb collisions is obtained in the rapidity window −1 < y < 0 and the pT-integrated yield dN/dy is extracted. It is found that the yields of protons, deuterons, and 3He, normalised by the spin degeneracy factor, follow an exponential decrease with mass number

Production of charged pions, kaons, and (anti-)protons in Pb-Pb and inelastic pppp collisions at sNN\sqrt {s_{NN}} = 5.02 TeV

International audienceMid-rapidity production of $\pi^{\pm}$, $\rm{K}^{\pm}$ and ($\bar{\rm{p}}$)p measured by the ALICE experiment at the LHC, in Pb-Pb and inelastic pp collisions at $\sqrt{s_{\rm{NN}}}$ = 5.02 TeV, is presented. The invariant yields are measured over a wide transverse momentum ($p_{\rm{T}}$) range from hundreds of MeV/$c$ up to 20 GeV/$c$. The results in Pb-Pb collisions are presented as a function of the collision centrality, in the range 0$-$90%. The comparison of the $p_{\rm{T}}$-integrated particle ratios, i.e. proton-to-pion (p/$\pi$) and kaon-to-pion (K/$\pi$) ratios, with similar measurements in Pb-Pb collisions at $\sqrt{s_{\rm{NN}}}$ = 2.76 TeV show no significant energy dependence. Blast-wave fits of the $p_{\rm{T}}$ spectra indicate that in the most central collisions radial flow is slightly larger at 5.02 TeV with respect to 2.76 TeV. Particle ratios (p/$\pi$, K/$\pi$) as a function of $p_{\rm{T}}$ show pronounced maxima at $p_{\rm{T}}$ $\approx$ 3 GeV/$c$ in central Pb-Pb collisions. At high $p_{\rm{T}}$, particle ratios at 5.02 TeV are similar to those measured in pp collisions at the same energy and in Pb-Pb collisions at $\sqrt{s_{\rm{NN}}}$ = 2.76 TeV. Using the pp reference spectra measured at the same collision energy of 5.02 TeV, the nuclear modification factors for the different particle species are derived. Within uncertainties, the nuclear modification factor is particle species independent for high $p_{\rm{T}}$ and compatible with measurements at $\sqrt{s_{\rm{NN}}}$ = 2.76 TeV. The results are compared to state-of-the-art model calculations, which are found to describe the observed trends satisfactorily

Multiplicity dependence of (multi-)strange hadron production in proton-proton collisions at s\sqrt{s} = 13 TeV

The production rates and the transverse momentum distribution of strange hadrons at mid-rapidity ($\left| y\right| < 0.5$) are measured in proton-proton collisions at $\sqrt{s}$ = 13 TeV as a function of the charged particle multiplicity, using the ALICE detector at the LHC. The production rates of $\mathrm{K}^{0}_{S}$, $\Lambda$, $\Xi$, and $\Omega$ increase with the multiplicity faster than what is reported for inclusive charged particles. The increase is found to be more pronounced for hadrons with a larger strangeness content. Possible auto-correlations between the charged particles and the strange hadrons are evaluated by measuring the event-activity with charged particle multiplicity estimators covering different pseudorapidity regions. When comparing to lower energy results, the yields of strange hadrons are found to depend only on the mid-rapidity charged particle multiplicity. Several features of the data are reproduced qualitatively by general purpose QCD Monte Carlo models that take into account the effect of densely-packed QCD strings in high multiplicity collisions. However, none of the tested models reproduce the data quantitatively. This work corroborates and extends the ALICE findings on strangeness production in proton-proton collisions at 7 TeV

Global polarization of ΛΛˉ\Lambda \bar \Lambda hyperons in Pb-Pb collisions at sNN\sqrt {s_{NN}} = 2.76 and 5.02 TeV

International audienceThe global polarization of the $\Lambda$ and $\overline\Lambda$ hyperons is measured for Pb-Pb collisions at $\sqrt{s_{\rm{NN}}}$ = 2.76 and 5.02 TeV recorded with the ALICE at the LHC. The results are reported differentially as a function of collision centrality and hyperon's transverse momentum ($p_{\rm{T}}$) for the range of centrality 5-50%, $0.5 < p_{\rm{T}} <5$ GeV/$c$, and rapidity $|y|<0.5$. The hyperon global polarization averaged for Pb-Pb collisions at $\sqrt{s_{\rm{NN}}}$ = 2.76 and 5.02 TeV is found to be consistent with zero, $\langle P_{\rm{H}}\rangle$ (%) $\approx$ 0.01 $\pm$ 0.06 (stat.) $\pm$ 0.03 (syst.) in the collision centrality range 15-50%, where the largest signal is expected. The results are compatible with expectations based on an extrapolation from measurements at lower collision energies at RHIC, hydrodynamical model calculations, and empirical estimates based on collision energy dependence of directed flow, all of which predict the global polarization values at LHC energies of the order of 0.01%

Λ3H^3_\Lambda\mathrm{H} and Λˉ3H‾^3_{\bar{\Lambda}}\mathrm{\overline{H}} lifetime measurement in Pb-Pb collisions at sNN=\sqrt{s_{\mathrm{NN}}} = 5.02 TeV via two-body decay

International audienceAn improved value for the lifetime of the (anti-)hypertriton has been obtained using the data sample of Pb–Pb collisions at sNN=5.02 TeV collected by the ALICE experiment at the LHC. The (anti-)hypertriton has been reconstructed via its charged two-body mesonic decay channel and the lifetime has been determined from an exponential fit to the d N /d( ct ) spectrum. The measured value, τ=242−38+34 (stat.) ± 17 (syst.) ps, is compatible with representative theoretical predictions, thus contributing to the solution of the longstanding hypertriton lifetime puzzle

Exploration of jet substructure using iterative declustering in pp and Pb–Pb collisions at LHC energies

The ALICE collaboration at the CERN LHC reports novel measurements of jet substructure in pp collisions at $\sqrt{s}$= 7 TeV and central Pb-Pb collisions at $\sqrt{s_{\rm{NN}}}$ = 2.76 TeV. Jet substructure of track-based jets is explored via iterative declustering and grooming techniques. We present the measurement of the momentum sharing of two-prong substructure exposed via grooming, the $z_{\rm{g}}$, and its dependence on the opening angle, in both pp and Pb-Pb collisions. We also present the first measurement of the distribution of the number of branches obtained in the iterative declustering of the jet, which is interpreted as the number of its hard splittings. In Pb-Pb collisions, we observe a suppression of symmetric splittings at large opening angles and an enhancement of splittings at small opening angles relative to pp collisions, with no significant modification of the number of splittings. The results are compared to predictions from various Monte Carlo event generators to test the role of important concepts in the evolution of the jet in the medium such as color coherence