Scaling Up Probabilistic Circuits by Latent Variable Distillation

Probabilistic Circuits (PCs) are a unified framework for tractable probabilistic models that support efficient computation of various probabilistic queries (e.g., marginal probabilities). One key challenge is to scale PCs to model large and high-dimensional real-world datasets: we observe that as the number of parameters in PCs increases, their performance immediately plateaus. This phenomenon suggests that the existing optimizers fail to exploit the full expressive power of large PCs. We propose to overcome such bottleneck by latent variable distillation: we leverage the less tractable but more expressive deep generative models to provide extra supervision over the latent variables of PCs. Specifically, we extract information from Transformer-based generative models to assign values to latent variables of PCs, providing guidance to PC optimizers. Experiments on both image and language modeling benchmarks (e.g., ImageNet and WikiText-2) show that latent variable distillation substantially boosts the performance of large PCs compared to their counterparts without latent variable distillation. In particular, on the image modeling benchmarks, PCs achieve competitive performance against some of the widely-used deep generative models, including variational autoencoders and flow-based models, opening up new avenues for tractable generative modeling

Mixtures of All Trees

Tree-shaped graphical models are widely used for their tractability. However, they unfortunately lack expressive power as they require committing to a particular sparse dependency structure. We propose a novel class of generative models called mixtures of all trees: that is, a mixture over all possible ($n^{n-2}$) tree-shaped graphical models over $n$ variables. We show that it is possible to parameterize this Mixture of All Trees (MoAT) model compactly (using a polynomial-size representation) in a way that allows for tractable likelihood computation and optimization via stochastic gradient descent. Furthermore, by leveraging the tractability of tree-shaped models, we devise fast-converging conditional sampling algorithms for approximate inference, even though our theoretical analysis suggests that exact computation of marginals in the MoAT model is NP-hard. Empirically, MoAT achieves state-of-the-art performance on density estimation benchmarks when compared against powerful probabilistic models including hidden Chow-Liu Trees.Comment: Accepted to AISTATS 202

Tractable Control for Autoregressive Language Generation

Despite the success of autoregressive large language models in text generation, it remains a major challenge to generate text that satisfies complex constraints: sampling from the conditional distribution $\Pr(\text{text} | \alpha)$ is intractable for even the simplest lexical constraints $\alpha$. To overcome this challenge, we propose to use tractable probabilistic models to impose lexical constraints in autoregressive text generation, which we refer to as GeLaTo. To demonstrate the effectiveness of this framework, we use distilled hidden Markov models to control autoregressive generation from GPT2. GeLaTo achieves state-of-the-art performance on CommonGen, a challenging benchmark for constrained text generation, beating a wide range of strong baselines by a large margin. Our work not only opens up new avenues for controlling large language models but also motivates the development of more expressive tractable probabilistic models.Comment: fixed typo in Table

Benthic Habitat Quality Assessment in Estuarine Intertidal Flats Based on Long-Term Data with Focus on Responses to Eco-Restoration Activity

A long-term assessment of the benthic habitat quality of intertidal flats in Liaohe Estuary was conducted by three integrating ecological indices, AZTI’s Marine Biotic Index (AMBI), Multivariate-AMBI (M-AMBI), and Shannon–Wiener diversity index (H′) based on macrobenthos data from 2013 to 2020. The results showed that the macrobenthic communities were characterized by indifferent and sensitive species of AMBI ecological groups. The annual ranges of H′, AMBI, and M-AMBI were 0.77–1.56, 1.44–3.73 and 0.36–0.54, respectively. Noticeable differences were found among assessment obtained by these biotic indices. Approximately 100%, 24%, and 78% sampling sites had “moderate”, “poor”, and “bad” statuses as assessed by H′, AMBI, and M-AMBI, respectively. Compared with H′ and AMBI, M-AMBI may be more applicable to evaluate the benthic habitat quality of intertidal flats in Liaohe Estuary. Results suggest that the benthic habitat quality in the middle parts of intertidal flats still had an unacceptable status and has not improved radically to date after large-scale “mariculture ponds restored to intertidal flats”.publishedVersio

High-frequency stimulation of nucleus accumbens changes in dopaminergic reward circuit

Deep brain stimulation (DBS) of the nucleus accumbens (NAc) is a potential remedial therapy for drug craving and relapse, but the mechanism is poorly understood. We investigated changes in neurotransmitter levels during high frequency stimulation (HFS) of the unilateral NAc on morphine-induced rats. Sixty adult Wistar rats were randomized into five groups: the control group (administration of saline), the morphine-only group (systematic administration of morphine without electrode implantation), the morphine-sham-stimulation group (systematic administration of morphine with electrode implantation but not given stimulation), the morphine-stimulation group (systematic administration of morphine with electrode implantation and stimulation) and the saline-stimulation group (administration of saline with electrode implantation and stimulation). The stimulation electrode was stereotaxically implanted into the core of unilateral NAc and microdialysis probes were unilaterally lowered into the ipsilateral ventral tegmental area (VTA), NAc, and ventral pallidum (VP). Samples from microdialysis probes in the ipsilateral VTA, NAc, and VP were analyzed for glutamate (Glu) and caminobutyric acid (GABA) by high-performance liquid chromatography (HPLC). The levels of Glu were increased in the ipsilateral NAc and VP of morphine-only group versus control group, whereas Glu levels were not significantly changed in the ipsilateral VTA. Furthermore, the levels of GABA decreased significantly in the ipsilateral NAc, VP, and VTA of morphineonly group when compared with control group. The profiles of increased Glu and reduced GABA in morphine-induced rats suggest that the presence of increased excitatory neurotransmission in these brain regions. The concentrations of the Glu significantly decreased while the levels of GABA increased in ipsilateral VTA, NAc, and VP in the morphine-stimulation group compared with the morphine-only group. No significant changes were seen in the morphine-sham stimulation group compared with the morphine-only group. These findings indicated that unilateral NAc stimulation inhibits the morphineinduced rats associated hyperactivation of excitatory neurotransmission in the mesocorticolimbic reward circuit

Community structure and plant diversity under different degrees of restored grassland in mining areas of the Qilian Mountains, Northwestern China

Background: Mining activities are known to exert significant effects on the structure and function of grassland ecosystems. However, the role of mining grasslands restoration in altering the plant community and soil quality remains poorly understood, especially in alpine regions. Here, we investigated species diversity in grasslands with dynamic changes and different restoration levels in the Tianzhu alpine mining area locating in the Qilian Mountains.Methods: The plant community structure and species composition of the grasslands with different restoration levels were analyzed by the sample method. We used five different restoration levels: very low recovered degree (VLRD), low recovered degree (LRD), medium recovered degree (MRD), and high recovered degree (HRD), and selected natural grassland (NGL, CK) as the control.Results: Plant community structure and species composition were significantly higher than those under the VLRD in the Tianzhu alpine mining area (p < 0.05), with HRD > MRD > LRD > VLRD. There were 11 families, 18 genera, and 17 species of plants, mainly in the families of Leguminosae, Asteraceae, Gramineae, Rosaceae, and Salicaceae; among them, Salicaceae and Gramineae played a decisive role in the stability of the community. The ecotype community showed that perennial herbaceous plants were the most dominant, with annual herbaceous plants being the least dominant, and no tree and shrub layers were observed; the dominance index was the highest in VLRD at 0.32, the richness index was the highest in HRD at 2.73, the diversity of HRD was higher at 1.93, soil pH and EC showed a decreasing trend, and SMC, SOC, TN, NO3-N, NH4-N, AN, TP, and AP content showed an increasing trend with the increase of grassland restoration.Conclusion: In summary, with the increase of restored grassland in the Tianzhu alpine mining area, plant diversity gradually increased and plant community structure gradually diversified, which was close to the plant diversity of NGL. The protection of partially VLRD and LRD grasslands in the mining area should be emphasized, and the mine grassland should be used rationally and scientifically restored

The dynamic effects of maternal high-calorie diet on glycolipid metabolism and gut microbiota from weaning to adulthood in offspring mice

Dysbiosis of gut microbiota can contribute to the progression of diabetes and obesity. Previous studies have shown that maternal high-fat (HF) diet during the perinatal period can alter the microbiota and induce metabolic disorders at weaning. However, whether dysbiosis of gut microbiota and metabolism could be recovered by a normal diet after weaning and the dynamic changes of gut microbiota have not been fully studied. In this study, C57BL/6J female mice were fed with a normal chow (NC) or HF diet for 4 weeks preconception, during gestation, and until pup weaning. After weaning, male offspring were fed with an NC diet until 9 weeks of age. The microbiota of offspring at weaning and 9 weeks of age was collected for 16S rRNA gene amplicon sequencing. We found that dams fed with an HF diet showed glucose intolerance after lactation. Compared with the offspring from NC dams, the offspring from HF dams exhibited a higher body weight, hyperglycemia, glucose intolerance, hyperinsulinemia, hypercholesterolemia, and leptin resistance and lower adiponectin at weaning. Fecal analysis indicated altered microbiota composition between the offspring of the two groups. The decrease in favorable bacteria (such as norank f Bacteroidales S24-7 group) and increase in unfavorable bacteria (such as Lachnoclostridium and Desulfovibrio) were strongly associated with a disturbance of glucose and lipid metabolism. After 6 weeks of normal diet, no difference in body weight, glucose, and lipid profiles was observed between the offspring of the two groups. However, the microbiota composition of offspring in the HF group was still different from that in the NC group, and microbiota diversity was lower in offspring of the HF group. The abundance of Lactobacillus was lower in the offspring of the HF group. In conclusion, a maternal HF diet can induce metabolic homeostasis and gut microbiota disturbance in offspring at weaning. Gut microbiota dysbiosis can persist into adulthood in the offspring, which might have a role in the promotion of susceptibility to obesity and diabetes in the later life of the offspring

Strain Distribution of Au and Ag Nanoparticles Embedded in Al 2

Au and Ag nanoparticles embedded in amorphous Al2O3 matrix are fabricated by the pulsed laser deposition (PLD) method and rapid thermal annealing (RTA) technique, which are confirmed by the experimental high-resolution transmission electron microscope (HRTEM) results, respectively. The strain distribution of Au and Ag nanoparticles embedded in the Al2O3 matrix is investigated by the finite-element (FE) calculations. The simulation results clearly indicate that both the Au and Ag nanoparticles incur compressive strain by the Al2O3 matrix. However, the compressive strain existing on the Au nanoparticle is much weaker than that on the Ag nanoparticle. This phenomenon can be attributed to the reason that Young’s modulus of Au is larger than that of Ag. This different strain distribution of Au and Ag nanoparticles in the same host matrix may have a significant influence on the technological potential applications of the Au-Ag alloy nanoparticles