Dense Text Retrieval based on Pretrained Language Models: A Survey

Text retrieval is a long-standing research topic on information seeking, where a system is required to return relevant information resources to user's queries in natural language. From classic retrieval methods to learning-based ranking functions, the underlying retrieval models have been continually evolved with the ever-lasting technical innovation. To design effective retrieval models, a key point lies in how to learn the text representation and model the relevance matching. The recent success of pretrained language models (PLMs) sheds light on developing more capable text retrieval approaches by leveraging the excellent modeling capacity of PLMs. With powerful PLMs, we can effectively learn the representations of queries and texts in the latent representation space, and further construct the semantic matching function between the dense vectors for relevance modeling. Such a retrieval approach is referred to as dense retrieval, since it employs dense vectors (a.k.a., embeddings) to represent the texts. Considering the rapid progress on dense retrieval, in this survey, we systematically review the recent advances on PLM-based dense retrieval. Different from previous surveys on dense retrieval, we take a new perspective to organize the related work by four major aspects, including architecture, training, indexing and integration, and summarize the mainstream techniques for each aspect. We thoroughly survey the literature, and include 300+ related reference papers on dense retrieval. To support our survey, we create a website for providing useful resources, and release a code repertory and toolkit for implementing dense retrieval models. This survey aims to provide a comprehensive, practical reference focused on the major progress for dense text retrieval

Specific-Heat Measurement of Residual Superconductivity in the Normal State of Underdoped Cuprate Superconductors

We have measured the magnetic field and temperature dependence of specific heat on $Bi_2Sr_{2-x}La_xCuO_{6+\delta}$ single crystals in wide doping and temperature regions. The superconductivity related specific heat coefficient $\gamma_{sc}$ and entropy $S_{sc}$ are determined. It is found that $\gamma_{sc}$ has a hump-like anomaly at $T_c$ and behaves as a long tail which persists far into the normal state for the underdoped samples, but for the heavily overdoped samples the anomaly ends sharply just near $T_c$. Interestingly, we found that the entropy associated with superconductivity is roughly conserved when and only the long tail part in the normal state is taken into account for the underdoped samples, indicating the residual superconductivity above T$_c$.Comment: 4 pages, 4 figures, Accepted for publication in Physical Review Letter

Full one-loop electroweak corrections to h0(H0,A0)H±W∓h^0(H^0,A^0) H^\pm W^\mp associated productions at e+e−e^+e^- linear colliders

We study the complete one-loop electroweak(EW) corrections to the processes of single charged Higgs boson production associated with a neutral Higgs boson$(h^0,H^0,A^0)$ and a gauge boson $W^\pm$ in the framework of the minimal supersymmetric standard model(MSSM). Numerical results at the ${\rm SPS1a'}$ benchmark point as proposed in the SPA project, are presented for demonstration. We find that for the process $e^+e^-\to h^0H^\pm W^\mp$ the EW relative correction can be either positive or negative and in the range of $-15$ in our chosen parameter space. While for the processes $e^+e^-\to H^0(A^0)H^\pm W^\mp$ the corrections generally reduce the Born cross sections and the EW relative corrections are typically of order $-10-20$.Comment: 22 pages, 20 figures, LaTex, to be appeared in PR

Correlation between diopters and refractive parameters among 5 to 12 years old school-age children of Lanzhou city

AIM: To investigate the epidemiological status of refractive state among school-age children of 5 to 12 years in Chengguan district of Lanzhou city, and to analyze the correlation with axial length(AL), horizontal and vertical corneal refractive power(K1, K2), anterior chamber depth(ACD)and corneal diameter(W-W). <p>METHODS: The value of AL, K1, K2, ACD and W-W of 813 school-age children(1626 eyes)aged 8.46±2.30 years old(5-12 years)were measured by IOL-Master, refractive error was measured by computer refractor. The eyes were divided into 5 groups according to different mean spherical equivalent(SE)diopter: high myopia, moderate myopia, low myopia, emmetropia and hyperopia. The correlation between diopter and different refractive parameters of different ages and different diopter groups were analyzed respectively, the SPSS 19.0 was used for the statistical analysis.<p>RESULTS: There were statistically significant difference(<i>P</i><0.05)in uncorrected visual acuity and refractive state among the different groups of 5-12 years school-age children, the prevalence of myopia tended to increase with the age. There were statistically significant difference in AL and ACD among the different age groups(<i>P</i><0.05), AL and ACD increased with the age. There were statistically significant difference in AL among three myopic groups(<i>P</i><0.05). There were statistically significant difference in ACD between low myopia and moderate myopia, low myopia and high myopia(<i>P</i><0.05). There were statistically significant difference in corneal refractive power and corneal diameter between low myopia and high myopia, moderate myopia and high myopia(<i>P</i><0.05). SE was negatively correlated with AL(<i>r</i>=-0.764, <i>P</i><0.01)and ACD(<i>r</i>=-0.498, <i>P</i><0.01), but had week correlation with corneal refractive power or corneal diameter.<p>CONCLUSION: The increase of AL plays an important role in the changes of refractive among school-age children of 5-12 years, AL is the main morphological variable related to myopia. Meanwhile, refractive changes may be affected by corneal refractive power(K1, K2), ACD and corneal diameter(W-W). 7-8 years old can be regarded as the main phase from emmetropization to myopization of school-age children

Critical Role of FoxO1 in Granulosa Cell Apoptosis Caused by Oxidative Stress and Protective Effects of Grape Seed Procyanidin B2

Reactive oxygen species (ROS) are closely related to the follicular granulosa cell apoptosis. Grape seed procyanidin B2 (GSPB2) has been reported to possess potent antioxidant activity. However, the GSPB2-mediated protective effects and the underlying molecular mechanisms in granulosa cell apoptosis process remain unknown. In this study, we showed for the first time that GSPB2 treatment decreased FoxO1 protein level, improved granulosa cell viability, upregulated LC3-II protein level, and reduced granulosa cell apoptosis rate. Under a condition of oxidative stress, GSPB2 reversed FoxO1 nuclear localization and increased its level in cytoplasm. In addition, FoxO1 knockdown inhibited the protective effects of GSPB2 induced. Our findings suggest that FoxO1 plays a pivotal role in regulating autophagy in granulosa cells, GSPB2 exerts a potent and beneficial role in reducing granulosa cell apoptosis and inducing autophagy process, and targeting FoxO1 could be significant in fighting against oxidative stress-reduced female reproductive system diseases

RocketQAv2: A Joint Training Method for Dense Passage Retrieval and Passage Re-ranking

In various natural language processing tasks, passage retrieval and passage re-ranking are two key procedures in finding and ranking relevant information. Since both the two procedures contribute to the final performance, it is important to jointly optimize them in order to achieve mutual improvement. In this paper, we propose a novel joint training approach for dense passage retrieval and passage re-ranking. A major contribution is that we introduce the dynamic listwise distillation, where we design a unified listwise training approach for both the retriever and the re-ranker. During the dynamic distillation, the retriever and the re-ranker can be adaptively improved according to each other's relevance information. We also propose a hybrid data augmentation strategy to construct diverse training instances for listwise training approach. Extensive experiments show the effectiveness of our approach on both MSMARCO and Natural Questions datasets. Our code is available at https://github.com/PaddlePaddle/RocketQA.Comment: EMNLP 202