RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling

Retrieval-augmented language models show promise in addressing issues like outdated information and hallucinations in language models (LMs). However, current research faces two main problems: 1) determining what information to retrieve, and 2) effectively combining retrieved information during generation. We argue that valuable retrieved information should not only be related to the current source text but also consider the future target text, given the nature of LMs that model future tokens. Moreover, we propose that aggregation using latent variables derived from a compact latent space is more efficient than utilizing explicit raw text, which is limited by context length and susceptible to noise. Therefore, we introduce RegaVAE, a retrieval-augmented language model built upon the variational auto-encoder (VAE). It encodes the text corpus into a latent space, capturing current and future information from both source and target text. Additionally, we leverage the VAE to initialize the latent space and adopt the probabilistic form of the retrieval generation paradigm by expanding the Gaussian prior distribution into a Gaussian mixture distribution. Theoretical analysis provides an optimizable upper bound for RegaVAE. Experimental results on various datasets demonstrate significant improvements in text generation quality and hallucination removal.Comment: Accepted to the Findings of EMNLP 202

IRRGN: An Implicit Relational Reasoning Graph Network for Multi-turn Response Selection

The task of response selection in multi-turn dialogue is to find the best option from all candidates. In order to improve the reasoning ability of the model, previous studies pay more attention to using explicit algorithms to model the dependencies between utterances, which are deterministic, limited and inflexible. In addition, few studies consider differences between the options before and after reasoning. In this paper, we propose an Implicit Relational Reasoning Graph Network to address these issues, which consists of the Utterance Relational Reasoner (URR) and the Option Dual Comparator (ODC). URR aims to implicitly extract dependencies between utterances, as well as utterances and options, and make reasoning with relational graph convolutional networks. ODC focuses on perceiving the difference between the options through dual comparison, which can eliminate the interference of the noise options. Experimental results on two multi-turn dialogue reasoning benchmark datasets MuTual and MuTual+ show that our method significantly improves the baseline of four pretrained language models and achieves state-of-the-art performance. The model surpasses human performance for the first time on the MuTual dataset.Comment: Accepted by EMNLP 202

Optimized Analysis Method for Evaluating the Shear Strength Parameters of Rock Joint Surfaces

The results obtained from the mechanical test of rock samples inevitably suffer dispersion owing to discrepancies between test specimens. In view of these deficiencies, the present study proposes a method based on the empirical equation of shear strength developed by Barton to determine the shear strength parameters of joint surfaces using a single test specimen. This approach is then applied to optimize the analysis of multiple specimens. An analysis of experimental results verifies that the shear strength parameters of joint surfaces obtained by the proposed method can more accurately reflect the shear mechanics of multiple specimens than conventional multiple sample analyses; meanwhile, the results are reasonable and reliable. More importantly, the optimized method ensures the shear strength parameters are no longer affected by the sequence of specimens employed during shear test. The optimized analysis method eliminates the effect of differences between specimens and the influence of subjective factors on test results and therefore provides more realistic evaluations of shear strength parameters

Oxygen-containing functional groups on bioelectrode surface enhance expression of c-type cytochromes in biofilm and boost extracellular electron transfer

Introducing oxygen-containing functional groups is a common and convenient method to increase the hydrophilicity of bioelectrodes. In this study, the effect of oxygen-containing functional groups on biofilm was systematically studied to understand how the electron transfer between electrochemically active bacteria (EAB) and bioelectrode was boosted. After electrolysis pretreatment in sulfuric and nitric acid mixture, the oxygen content of the carbon fiber brushes increased from 4.6% to 30.9%. Comparing with the control, the maximum power density increased by 27.7%, while the anode resistance decreased by 21.8%, because charge transfer resistance significantly reduced. The analysis results showed that the content of c-type cytochromes (c-Cyts) in the EAB biofilm was four times higher than that in the control, while the biomass just slightly increased and the bacteria community was similar with that of the control. These findings suggested that the fundamental reason for the enhanced extracellular electron transfer between EAB and electrode was the increased c-Cyts

A Preliminary Study on Realizing Human–Robot Mental Comforting Dialogue via Sharing Experience Emotionally

Mental health issues are receiving more and more attention in society. In this paper, we introduce a preliminary study on human–robot mental comforting conversation, to make an android robot (ERICA) present an understanding of the user’s situation by sharing similar emotional experiences to enhance the perception of empathy. Specifically, we create the emotional speech for ERICA by using CycleGAN-based emotional voice conversion model, in which the pitch and spectrogram of the speech are converted according to the user’s mental state. Then, we design dialogue scenarios for the user to talk about his/her predicament with ERICA. In the dialogue, ERICA shares other people’s similar predicaments and adopts a low-spirit voice to express empathy to the interlocutor’s situation. At the end of the dialogue, ERICA tries to encourage with a positive voice. Subsequently, questionnaire-based evaluation experiments were conducted with the recorded conversation. In the questionnaire, we use the Big Five scale to evaluate ERICA’s personality. In addition, the perception of emotion, empathy, and encouragement in the dialogue are evaluated. The results show that the proposed emotional expression strategy helps the android robot better present low-spirit emotion, empathy, the personality of extroversion, while making the user better feel the encouragement

Analysis of Changes in the Micromorphology of Sandstone Joint Surface under Dry-Wet Cycling

Changes in the micromorphology of joint surface under dry-wet cycling have a direct effect on the mechanical properties of the jointed rock masses, which in turn affects the deformation stability of the bank slope of a reservoir. In this study, we design and carry out a test that aims to quantity the effects of repeated rise and fall of a reservoir on the properties of a jointed rock masses. The results are as follows: first, the roughness, local gradient, and undulation of the joint surface gradually decreased under repeated dry-wet cycling. In addition, the height parameters and texture parameters showed a steep decrease followed by a slow decline. The deterioration was particularly obvious over the first 5 dry-wet cycles. Second, the roughness coefficient of the joint surface, the compressive strength of the face wall, and the basic friction angle were gradually reduced under dry-wet cycling. The shear strength of the jointed rock masses (obtained both quantitatively and experimentally) showed a deteriorating trend controlled by the deterioration of the micromorphology, the strength of the face wall, and the frictional properties of the joint surface. Finally, the dry-wet cycling process determined trends of changes in the microstructure parameters and mechanical properties of the joint surface. Our research results provide a good basis for the analysis of the deterioration and failure of rock masses within the hydrofluctuation belt of a bank slope

Dynamic sediment discharge in the Hekou-Longmen region of Yellow River and soil and water conservation implications

The middle reaches of the Yellow River Basin transport the vast majority of sediment (>85% of the basin's total available sediment load), which has had profound effects on the characteristics of the middle and lower reaches of the Yellow River. Since the late 1950s, soil and water conservation measures have been extensively implemented in the Loess Plateau, China, especially since the 1970s. This has resulted in sediment discharge changing significantly. In this study, data from 22 catchments in the region of the Loess Plateau from Hekou to Longmen in the middle reaches of the Yellow River were analyzed to investigate the responses of the sediment regime to climate change and human activities. The non-parametric Mann-Kendall test and the Pettitt test were used to identify trends and shifts in sediment discharge. All 22 catchments had a significantly decreasing trend (P < 0.01) in annual sediment discharge. Change point years were detected between 1971 and 1994, and were concentrated between 1978 and 1984 in 17 catchments. Moreover, erosive rainfall exhibited a tendency to decrease, but this was not a significant trend. Compared to rainfall, human activities, primarily soil and water conservation and environmental rehabilitation campaigns, have played a more prominent role in the changes in sediment regimes. In order to reduce soil erosion and sediment yield, more attention should be paid to proper and rational soil and water conservation and eco-restoration in this region. (C) 2016 Elsevier B.V. All rights reserved

Enhanced osteoinductivity and corrosion resistance of dopamine/gelatin/rhBMP-2-coated β-TCP/Mg-Zn orthopedic implants: An in vitro and in vivo study.

Magnesium-based biomaterials are attracting increasingly more attention for orthopedic applications based on their appropriate mechanical properties, biodegradability, and favorable biocompatibility. However, the high corrosion rate of these materials remains to be addressed. In this study, porous β-Ca3(PO4)2/Mg-Zn (β-TCP/Mg-Zn) composites were fabricated via a powder metallurgy method. The β-TCP/Mg-Zn composites with 6% porosity exhibited optimal mechanical properties, and thus, they were selected for surface modification. A novel dopamine/gelatin/recombinant human bone morphogenetic protein-2 (rhBMP-2) coating with demonstrated stability was prepared to further improve the corrosion resistance of the composite and enhance early osteoinductivity. The homogeneously coated β-TCP/Mg-Zn composite showed significantly improved corrosion resistance according to electrochemical and immersion tests. In addition, extracts from the dopamine/gelatin/rhBMP-2-coated β-TCP/Mg-Zn composite not only facilitated cell proliferation but also significantly enhanced the osteogenic differentiation of Sprague-Dawley rat bone marrow-derived mesenchymal stem cells in vitro. Furthermore, in vivo experiments were performed to evaluate the biodegradation, histocompatibility, and osteoinductive potential of the coated composite. No obvious pathological changes in the vital visceral organs were observed after implantation, and radiography and hematoxylin-eosin staining showed strong promotion of new bone formation, matched composite degradation and bone regeneration rates, and complete absorption of the released hydrogen gas. Collectively, these results indicate that the dopamine/gelatin/rhBMP-2-coated β-TCP/Mg-Zn composite offers improved corrosion resistance, favorable biocompatibility, and enhanced osteoinductive potential for use in the fabrication of orthopedic implants

The coupling of multi-channel optical vortices based on angular momentum conservation using a single-layer metal metasurface

Optical vortices (OVs) carry the orbital angular momentum with arbitrary topological charges, which has excellent potential in optical communication, photonic integrated circuits, optical trapping, and so on. However, generating arbitrary orders of adjustable optical vortices remains to be solved. Here, we propose a single-layer metal porous metasurface operating in infrared band for generating vortex beams from first to fourth order based on the spin-orbit interactions (SOI). The optical vortices with integral 2π phase are obtained through generating double geometric phase induced by structural element spin rotation. Furthermore, the new phenomenon of optical vortices emerging on the center has also been observed in our system, which is caused by the coupling of multi-channel same-order OVs. Our work possesses wide applications in optical communication, multiplex and demultiplex systems, optical capture devices, and communication coding