Learning an Effective Context-Response Matching Model with Self-Supervised Tasks for Retrieval-based Dialogues

Building an intelligent dialogue system with the ability to select a proper response according to a multi-turn context is a great challenging task. Existing studies focus on building a context-response matching model with various neural architectures or PLMs and typically learning with a single response prediction task. These approaches overlook many potential training signals contained in dialogue data, which might be beneficial for context understanding and produce better features for response prediction. Besides, the response retrieved from existing dialogue systems supervised by the conventional way still faces some critical challenges, including incoherence and inconsistency. To address these issues, in this paper, we propose learning a context-response matching model with auxiliary self-supervised tasks designed for the dialogue data based on pre-trained language models. Specifically, we introduce four self-supervised tasks including next session prediction, utterance restoration, incoherence detection and consistency discrimination, and jointly train the PLM-based response selection model with these auxiliary tasks in a multi-task manner. By this means, the auxiliary tasks can guide the learning of the matching model to achieve a better local optimum and select a more proper response. Experiment results on two benchmarks indicate that the proposed auxiliary self-supervised tasks bring significant improvement for multi-turn response selection in retrieval-based dialogues, and our model achieves new state-of-the-art results on both datasets.Comment: 10 page

Preparation of graphene film reinforced CoCrFeNiMn high-entropy alloy matrix composites with strength-plasticity synergy via flake powder metallurgy method

Inspired by the design principle of pearl structure, a bottom-up flake powder self-assembly arrangement strategy, flake powder metallurgy, is used to prepare graphene films (GFs) reinforced CoCrFeNiMn high-entropy alloy (HEA) matrix composites with a pearl laminated structure. Flaky HEA powder was prepared by ball milling method and homogeneously mixed with Ni plated GFs. Vacuum hot-press sintering (VHPS) technique was carried out to solidify the mixed powders to obtain composites with uniform distribution of GFs(Ni) and flaky HEA. The results show that the bottom-up preparation strategy can effectively fabricate bionic laminated HEA matrix composites, and the composites have a distinct pearly laminated structure. The tensile strength of the composites with 5 vol% GFs(Ni) content reached 834.04 MPa, and the elongation reached 26.58 %. The compressive strength in parallel and perpendicular laminar directions reached 2069.66 MPa and 2418.45 MPa at 50 % strain, respectively. The laminated GFs(Ni)/HEA matrix composites possessed excellent strength and maintained good plasticity. In this study, the strengthening and toughening mechanism of the laminated GFs(Ni)/HEA matrix composites is discussed in detail, and the results show that the laminated structure and GFs(Ni) are favorable for the hardening and strengthening of the HEA matrix

WizardLM: Empowering Large Language Models to Follow Complex Instructions

Training large language models (LLM) with open-domain instruction following data brings colossal success. However, manually creating such instruction data is very time-consuming and labor-intensive. Moreover, humans may struggle to produce high-complexity instructions. In this paper, we show an avenue for creating large amounts of instruction data with varying levels of complexity using LLM instead of humans. Starting with an initial set of instructions, we use our proposed Evol-Instruct to rewrite them step by step into more complex instructions. Then, we mix all generated instruction data to fine-tune LLaMA. We call the resulting model WizardLM. Human evaluations on a complexity-balanced test bed show that instructions from Evol-Instruct are superior to human-created ones. By analyzing the human evaluation results of the high complexity part, we demonstrate that outputs from our WizardLM model are preferred to outputs from OpenAI ChatGPT. Even though WizardLM still lags behind ChatGPT in some aspects, our findings suggest that fine-tuning with AI-evolved instructions is a promising direction for enhancing large language models. Our codes and generated data are public at https://github.com/nlpxucan/WizardLMComment: large language model, instruction fine-tun

LexMAE: Lexicon-Bottlenecked Pretraining for Large-Scale Retrieval

In large-scale retrieval, the lexicon-weighting paradigm, learning weighted sparse representations in vocabulary space, has shown promising results with high quality and low latency. Despite it deeply exploiting the lexicon-representing capability of pre-trained language models, a crucial gap remains between language modeling and lexicon-weighting retrieval -- the former preferring certain or low-entropy words whereas the latter favoring pivot or high-entropy words -- becoming the main barrier to lexicon-weighting performance for large-scale retrieval. To bridge this gap, we propose a brand-new pre-training framework, lexicon-bottlenecked masked autoencoder (LexMAE), to learn importance-aware lexicon representations. Essentially, we present a lexicon-bottlenecked module between a normal language modeling encoder and a weakened decoder, where a continuous bag-of-words bottleneck is constructed to learn a lexicon-importance distribution in an unsupervised fashion. The pre-trained LexMAE is readily transferred to the lexicon-weighting retrieval via fine-tuning. On the ad-hoc retrieval benchmark, MS-Marco, it achieves 42.6% MRR@10 with 45.8 QPS for the passage dataset and 44.4% MRR@100 with 134.8 QPS for the document dataset, by a CPU machine. And LexMAE shows state-of-the-art zero-shot transfer capability on BEIR benchmark with 12 datasets.Comment: Appeared at ICLR 202

Synergistic Interplay between Search and Large Language Models for Information Retrieval

Information retrieval (IR) plays a crucial role in locating relevant resources from vast amounts of data, and its applications have evolved from traditional knowledge bases to modern retrieval models (RMs). The emergence of large language models (LLMs) has further revolutionized the IR field by enabling users to interact with search systems in natural languages. In this paper, we explore the advantages and disadvantages of LLMs and RMs, highlighting their respective strengths in understanding user-issued queries and retrieving up-to-date information. To leverage the benefits of both paradigms while circumventing their limitations, we propose InteR, a novel framework that facilitates information refinement through synergy between RMs and LLMs. InteR allows RMs to expand knowledge in queries using LLM-generated knowledge collections and enables LLMs to enhance prompt formulation using retrieved documents. This iterative refinement process augments the inputs of RMs and LLMs, leading to more accurate retrieval. Experiments on large-scale retrieval benchmarks involving web search and low-resource retrieval tasks demonstrate that InteR achieves overall superior zero-shot retrieval performance compared to state-of-the-art methods, even those using relevance judgment. Source code is available at https://github.com/Cyril-JZ/InteRComment: Pre-print. Work in progres

A New Unified Solution for Deep Tunnels in Water-Rich Areas considering Pore Water Pressure

Pore water pressure has an important influence on the stresses and deformation of the surrounding rock of deep tunnels in water-rich areas. In this study, a mechanical model for deep tunnels subjected to a nonuniform stress field in water-rich areas is developed. Considering the pore water pressure, a new unified solution for the stresses, postpeak zone radii, and surface displacement is derived based on a strain-softening model and the Mogi-Coulomb criterion. Through a case study, the effects of pore water pressure, intermediate principal stress, and residual cohesion on the stress distribution, postpeak zone radii, and surface displacement are also discussed. Results show that the tangential stresses are always larger than the radial stress. The radial stress presents a gradually increasing trend, while the tangential stress presents a trend of first increasing and then decreasing, and the maximum tangential stress appears at the interface between the elastic and plastic zones. As the pore water pressure increases, the postpeak zone radii and surface displacement increase. Because of the neglect of the intermediate principal stress in the Mohr-Coulomb criterion, the postpeak zone radii, surface displacement, and maximum tangential stress solved by the Mohr-Coulomb criterion are all larger than those solved by the Mogi-Coulomb criterion. Tunnels surrounded by rock masses with a higher residual cohesion experience lower postpeak zone radii and surface displacement. Data presented in this study provide an important theoretical basis for supporting the tunnels in water-rich areas

The Influence of Tip Clearance on the Performance of a High-Speed Inducer Centrifugal Pump under Different Flow Rates Conditions

The influence mechanism of the blade tip clearance (TC) of an inducer on the performance of a centrifugal pump at high speed was researched under different flow rate conditions in this work. An experiment on the pump’s external performance was carried out, and numerical calculation was also performed under four different TCs. The full characteristic performance curves, static pressure and pressure pulsation distributions of the pump were obtained. Through the research and analysis, it was found that the influence of the TC on the efficiency and the head of the centrifugal pump are related to the flow rate. Under the influence of a large flow rate, the increase in the TC is helpful to improve the efficiency and the head of the pump. The increase in the TC helps to weaken the gap jet effect on the inducer. The inlet jet of the inducer, caused by TC leakage, will form a low-pressure vortex zone at the inlet of the inducer. The splitter-bladed inducer’s pressure pulsation is affected by the TC. The peak pressure pulsation at the monitoring point at the short blades is larger than that at the long blades. With the increase in TC, the cavitation degree at the inlet of the long blade of the inducer is decreased, while the cavitation degree at the short blade is deepened. It is also found that the TC has little effect on the radial force of the inducer and the impeller. These results will provide the design basis for the tip clearance of an inducer

How Safety Climate Impacts Safety Voice—Investigating the Mediating Role of Psychological Safety from a Social Cognitive Perspective

Safety voice has become a popular research topic in the organizational safety field because it helps to prevent accidents. A good safety climate and psychological safety can motivate employees to actively express their ideas about safety, but the specific mechanisms of safety climate and psychological safety, on safety voice, are not yet clear. Based on the “environment-subject cognition-behavior” triadic interaction model of social cognitive theory, this paper explores the relationship between safety climate and safety voice, and the mediating role of psychological safety. We collected questionnaires and conducted data analysis of the valid questionnaires using analytical methods such as hierarchical regression, stepwise regression, and the bootstrap sampling method. We found that safety climate significantly and positively influenced safety voice, and psychological safety played a mediating role between safety climate and safety voice, which strengthened the positive relationship between them. From the research results, it was clear that to stimulate employees to express safety voice behavior, organizations should strive to create a good safety climate and pay attention to building employees’ psychological safety. The findings of this paper provide useful insights for the management of employee safety voice behavior in enterprises