26 research outputs found
RegaVAE: A Retrieval-Augmented Gaussian Mixture Variational Auto-Encoder for Language Modeling
Retrieval-augmented language models show promise in addressing issues like
outdated information and hallucinations in language models (LMs). However,
current research faces two main problems: 1) determining what information to
retrieve, and 2) effectively combining retrieved information during generation.
We argue that valuable retrieved information should not only be related to the
current source text but also consider the future target text, given the nature
of LMs that model future tokens. Moreover, we propose that aggregation using
latent variables derived from a compact latent space is more efficient than
utilizing explicit raw text, which is limited by context length and susceptible
to noise. Therefore, we introduce RegaVAE, a retrieval-augmented language model
built upon the variational auto-encoder (VAE). It encodes the text corpus into
a latent space, capturing current and future information from both source and
target text. Additionally, we leverage the VAE to initialize the latent space
and adopt the probabilistic form of the retrieval generation paradigm by
expanding the Gaussian prior distribution into a Gaussian mixture distribution.
Theoretical analysis provides an optimizable upper bound for RegaVAE.
Experimental results on various datasets demonstrate significant improvements
in text generation quality and hallucination removal.Comment: Accepted to the Findings of EMNLP 202
IRRGN: An Implicit Relational Reasoning Graph Network for Multi-turn Response Selection
The task of response selection in multi-turn dialogue is to find the best
option from all candidates. In order to improve the reasoning ability of the
model, previous studies pay more attention to using explicit algorithms to
model the dependencies between utterances, which are deterministic, limited and
inflexible. In addition, few studies consider differences between the options
before and after reasoning. In this paper, we propose an Implicit Relational
Reasoning Graph Network to address these issues, which consists of the
Utterance Relational Reasoner (URR) and the Option Dual Comparator (ODC). URR
aims to implicitly extract dependencies between utterances, as well as
utterances and options, and make reasoning with relational graph convolutional
networks. ODC focuses on perceiving the difference between the options through
dual comparison, which can eliminate the interference of the noise options.
Experimental results on two multi-turn dialogue reasoning benchmark datasets
MuTual and MuTual+ show that our method significantly improves the baseline of
four pretrained language models and achieves state-of-the-art performance. The
model surpasses human performance for the first time on the MuTual dataset.Comment: Accepted by EMNLP 202
Stable Knowledge Editing in Large Language Models
Efficient knowledge editing of large language models is crucial for replacing
obsolete information or incorporating specialized knowledge on a large scale.
However, previous methods implicitly assume that knowledge is localized and
isolated within the model, an assumption that oversimplifies the interconnected
nature of model knowledge. The premise of localization results in an incomplete
knowledge editing, whereas an isolated assumption may impair both other
knowledge and general abilities. It introduces instability to the performance
of the knowledge editing method. To transcend these assumptions, we introduce
StableKE, a method adopts a novel perspective based on knowledge augmentation
rather than knowledge localization. To overcome the expense of human labeling,
StableKE integrates two automated knowledge augmentation strategies: Semantic
Paraphrase Enhancement strategy, which diversifies knowledge descriptions to
facilitate the teaching of new information to the model, and Contextual
Description Enrichment strategy, expanding the surrounding knowledge to prevent
the forgetting of related information. StableKE surpasses other knowledge
editing methods, demonstrating stability both edited knowledge and multi-hop
knowledge, while also preserving unrelated knowledge and general abilities.
Moreover, StableKE can edit knowledge on ChatGPT
Optimized Analysis Method for Evaluating the Shear Strength Parameters of Rock Joint Surfaces
The results obtained from the mechanical test of rock samples inevitably suffer dispersion owing to discrepancies between test specimens. In view of these deficiencies, the present study proposes a method based on the empirical equation of shear strength developed by Barton to determine the shear strength parameters of joint surfaces using a single test specimen. This approach is then applied to optimize the analysis of multiple specimens. An analysis of experimental results verifies that the shear strength parameters of joint surfaces obtained by the proposed method can more accurately reflect the shear mechanics of multiple specimens than conventional multiple sample analyses; meanwhile, the results are reasonable and reliable. More importantly, the optimized method ensures the shear strength parameters are no longer affected by the sequence of specimens employed during shear test. The optimized analysis method eliminates the effect of differences between specimens and the influence of subjective factors on test results and therefore provides more realistic evaluations of shear strength parameters
Oxygen-containing functional groups on bioelectrode surface enhance expression of c-type cytochromes in biofilm and boost extracellular electron transfer
Introducing oxygen-containing functional groups is a common and convenient method to increase the hydrophilicity of bioelectrodes. In this study, the effect of oxygen-containing functional groups on biofilm was systematically studied to understand how the electron transfer between electrochemically active bacteria (EAB) and bioelectrode was boosted. After electrolysis pretreatment in sulfuric and nitric acid mixture, the oxygen content of the carbon fiber brushes increased from 4.6% to 30.9%. Comparing with the control, the maximum power density increased by 27.7%, while the anode resistance decreased by 21.8%, because charge transfer resistance significantly reduced. The analysis results showed that the content of c-type cytochromes (c-Cyts) in the EAB biofilm was four times higher than that in the control, while the biomass just slightly increased and the bacteria community was similar with that of the control. These findings suggested that the fundamental reason for the enhanced extracellular electron transfer between EAB and electrode was the increased c-Cyts
A Preliminary Study on Realizing Human–Robot Mental Comforting Dialogue via Sharing Experience Emotionally
Mental health issues are receiving more and more attention in society. In this paper, we introduce a preliminary study on human–robot mental comforting conversation, to make an android robot (ERICA) present an understanding of the user’s situation by sharing similar emotional experiences to enhance the perception of empathy. Specifically, we create the emotional speech for ERICA by using CycleGAN-based emotional voice conversion model, in which the pitch and spectrogram of the speech are converted according to the user’s mental state. Then, we design dialogue scenarios for the user to talk about his/her predicament with ERICA. In the dialogue, ERICA shares other people’s similar predicaments and adopts a low-spirit voice to express empathy to the interlocutor’s situation. At the end of the dialogue, ERICA tries to encourage with a positive voice. Subsequently, questionnaire-based evaluation experiments were conducted with the recorded conversation. In the questionnaire, we use the Big Five scale to evaluate ERICA’s personality. In addition, the perception of emotion, empathy, and encouragement in the dialogue are evaluated. The results show that the proposed emotional expression strategy helps the android robot better present low-spirit emotion, empathy, the personality of extroversion, while making the user better feel the encouragement
Analysis of Changes in the Micromorphology of Sandstone Joint Surface under Dry-Wet Cycling
Changes in the micromorphology of joint surface under dry-wet cycling have a direct effect on the mechanical properties of the jointed rock masses, which in turn affects the deformation stability of the bank slope of a reservoir. In this study, we design and carry out a test that aims to quantity the effects of repeated rise and fall of a reservoir on the properties of a jointed rock masses. The results are as follows: first, the roughness, local gradient, and undulation of the joint surface gradually decreased under repeated dry-wet cycling. In addition, the height parameters and texture parameters showed a steep decrease followed by a slow decline. The deterioration was particularly obvious over the first 5 dry-wet cycles. Second, the roughness coefficient of the joint surface, the compressive strength of the face wall, and the basic friction angle were gradually reduced under dry-wet cycling. The shear strength of the jointed rock masses (obtained both quantitatively and experimentally) showed a deteriorating trend controlled by the deterioration of the micromorphology, the strength of the face wall, and the frictional properties of the joint surface. Finally, the dry-wet cycling process determined trends of changes in the microstructure parameters and mechanical properties of the joint surface. Our research results provide a good basis for the analysis of the deterioration and failure of rock masses within the hydrofluctuation belt of a bank slope
Dynamic sediment discharge in the Hekou-Longmen region of Yellow River and soil and water conservation implications
The middle reaches of the Yellow River Basin transport the vast majority of sediment (>85% of the basin's total available sediment load), which has had profound effects on the characteristics of the middle and lower reaches of the Yellow River. Since the late 1950s, soil and water conservation measures have been extensively implemented in the Loess Plateau, China, especially since the 1970s. This has resulted in sediment discharge changing significantly. In this study, data from 22 catchments in the region of the Loess Plateau from Hekou to Longmen in the middle reaches of the Yellow River were analyzed to investigate the responses of the sediment regime to climate change and human activities. The non-parametric Mann-Kendall test and the Pettitt test were used to identify trends and shifts in sediment discharge. All 22 catchments had a significantly decreasing trend (P < 0.01) in annual sediment discharge. Change point years were detected between 1971 and 1994, and were concentrated between 1978 and 1984 in 17 catchments. Moreover, erosive rainfall exhibited a tendency to decrease, but this was not a significant trend. Compared to rainfall, human activities, primarily soil and water conservation and environmental rehabilitation campaigns, have played a more prominent role in the changes in sediment regimes. In order to reduce soil erosion and sediment yield, more attention should be paid to proper and rational soil and water conservation and eco-restoration in this region. (C) 2016 Elsevier B.V. All rights reserved
Enhanced osteoinductivity and corrosion resistance of dopamine/gelatin/rhBMP-2-coated β-TCP/Mg-Zn orthopedic implants: An in vitro and in vivo study.
Magnesium-based biomaterials are attracting increasingly more attention for orthopedic applications based on their appropriate mechanical properties, biodegradability, and favorable biocompatibility. However, the high corrosion rate of these materials remains to be addressed. In this study, porous β-Ca3(PO4)2/Mg-Zn (β-TCP/Mg-Zn) composites were fabricated via a powder metallurgy method. The β-TCP/Mg-Zn composites with 6% porosity exhibited optimal mechanical properties, and thus, they were selected for surface modification. A novel dopamine/gelatin/recombinant human bone morphogenetic protein-2 (rhBMP-2) coating with demonstrated stability was prepared to further improve the corrosion resistance of the composite and enhance early osteoinductivity. The homogeneously coated β-TCP/Mg-Zn composite showed significantly improved corrosion resistance according to electrochemical and immersion tests. In addition, extracts from the dopamine/gelatin/rhBMP-2-coated β-TCP/Mg-Zn composite not only facilitated cell proliferation but also significantly enhanced the osteogenic differentiation of Sprague-Dawley rat bone marrow-derived mesenchymal stem cells in vitro. Furthermore, in vivo experiments were performed to evaluate the biodegradation, histocompatibility, and osteoinductive potential of the coated composite. No obvious pathological changes in the vital visceral organs were observed after implantation, and radiography and hematoxylin-eosin staining showed strong promotion of new bone formation, matched composite degradation and bone regeneration rates, and complete absorption of the released hydrogen gas. Collectively, these results indicate that the dopamine/gelatin/rhBMP-2-coated β-TCP/Mg-Zn composite offers improved corrosion resistance, favorable biocompatibility, and enhanced osteoinductive potential for use in the fabrication of orthopedic implants