ReSLLM: Large Language Models are Strong Resource Selectors for Federated Search

Federated search, which involves integrating results from multiple independent search engines, will become increasingly pivotal in the context of Retrieval-Augmented Generation pipelines empowering LLM-based applications such as chatbots. These systems often distribute queries among various search engines, ranging from specialized (e.g., PubMed) to general (e.g., Google), based on the nature of user utterances. A critical aspect of federated search is resource selection - the selection of appropriate resources prior to issuing the query to ensure high-quality and rapid responses, and contain costs associated with calling the external search engines. However, current SOTA resource selection methodologies primarily rely on feature-based learning approaches. These methods often involve the labour intensive and expensive creation of training labels for each resource. In contrast, LLMs have exhibited strong effectiveness as zero-shot methods across NLP and IR tasks. We hypothesise that in the context of federated search LLMs can assess the relevance of resources without the need for extensive predefined labels or features. In this paper, we propose ReSLLM. Our ReSLLM method exploits LLMs to drive the selection of resources in federated search in a zero-shot setting. In addition, we devise an unsupervised fine tuning protocol, the Synthetic Label Augmentation Tuning (SLAT), where the relevance of previously logged queries and snippets from resources is predicted using an off-the-shelf LLM and then in turn used to fine-tune ReSLLM with respect to resource selection. Our empirical evaluation and analysis details the factors influencing the effectiveness of LLMs in this context. The results showcase the merits of ReSLLM for resource selection: not only competitive effectiveness in the zero-shot setting, but also obtaining large when fine-tuned using SLAT-protocol

Selecting which Dense Retriever to use for Zero-Shot Search

We propose the new problem of choosing which dense retrieval model to use when searching on a new collection for which no labels are available, i.e. in a zero-shot setting. Many dense retrieval models are readily available. Each model however is characterized by very differing search effectiveness -- not just on the test portion of the datasets in which the dense representations have been learned but, importantly, also across different datasets for which data was not used to learn the dense representations. This is because dense retrievers typically require training on a large amount of labeled data to achieve satisfactory search effectiveness in a specific dataset or domain. Moreover, effectiveness gains obtained by dense retrievers on datasets for which they are able to observe labels during training, do not necessarily generalise to datasets that have not been observed during training. This is however a hard problem: through empirical experimentation we show that methods inspired by recent work in unsupervised performance evaluation with the presence of domain shift in the area of computer vision and machine learning are not effective for choosing highly performing dense retrievers in our setup. The availability of reliable methods for the selection of dense retrieval models in zero-shot settings that do not require the collection of labels for evaluation would allow to streamline the widespread adoption of dense retrieval. This is therefore an important new problem we believe the information retrieval community should consider. Implementation of methods, along with raw result files and analysis scripts are made publicly available at https://www.github.com/anonymized

Weight of Evidence Method and Its Applications and Development

AbstractThe development and applications about the weight of evidence technology in recent years are reviewed. This paper introduced the improved weight of evidence in remote sensing image processing and in different fields of application. Summary its constraints and existent problems. Look forward to the weight of evidence for the practical application

Zero-shot Generative Large Language Models for Systematic Review Screening Automation

Systematic reviews are crucial for evidence-based medicine as they comprehensively analyse published research findings on specific questions. Conducting such reviews is often resource- and time-intensive, especially in the screening phase, where abstracts of publications are assessed for inclusion in a review. This study investigates the effectiveness of using zero-shot large language models~(LLMs) for automatic screening. We evaluate the effectiveness of eight different LLMs and investigate a calibration technique that uses a predefined recall threshold to determine whether a publication should be included in a systematic review. Our comprehensive evaluation using five standard test collections shows that instruction fine-tuning plays an important role in screening, that calibration renders LLMs practical for achieving a targeted recall, and that combining both with an ensemble of zero-shot models saves significant screening time compared to state-of-the-art approaches.Comment: Accepted to ECIR2024 full paper (findings

Phase diagram of superconducting vortex ratchet motion in a superlattice with noncentrosymmetry

Ratchet motion of superconducting vortices, which is a directional flow of vortices in superconductors, is highly useful for exploring quantum phenomena and developing superconducting devices, such as superconducting diode and microwave antenna. However, because of the challenges in the quantitative characterization of the dynamic motion of vortices, a phase diagram of the vortex ratchet motion is still missing, especially in the superconductors with low dimensional structures. Here we establish a quantitative phase diagram of the vortex ratchet motion in a highly anisotropic superlattice superconductor, (SnS)1.17NbS2, using nonreciprocal magnetotransport. The (SnS)1.17NbS2, which possesses a layered atomic structure and noncentrosymmetry, exhibits nonreciprocal magnetotransport in a magnetic field perpendicular and parallel to the plane, which is considered a manifest of ratchet motion of superconducting vortices. We demonstrated that the ratchet motion is responsive to current excitation, magnetic field and thermal perturbation. Furthermore, we extrapolated a giant nonreciprocal coefficient ({\gamma}), which quantitatively describes the magnitude of the vortex ratchet motion, and eventually established phase diagrams of the ratchet motion of the vortices with a quantitative description. Last, we propose that the ratchet motion originates from the coexistence of pancake vortices (PVs) and Josephson vortices (JVs). The phase diagrams are desirable for controlling the vortex motion in superlattice superconductors and developing next-generation energy-efficient superconducting devices

Submarine groundwater discharge in Dongshan Bay, China: A master regulator of nutrients in spring and potential national significance of small bays

Despite over 90% of China’s coastal bays have an area less than 500 km2, the geochemical effects of SGD on those ecosystems are ambiguous. Based on mapping and time-series observations of Ra isotopes and nutrients, a case study of small bays (<500 km2), we revealed that submarine groundwater discharge (SGD) predominately regulated the distribution of nutrients and fueled algal growth in Dongshan Bay, China. On the bay-wide scale, the SGD rate was estimated to be 0.048 ± 0.022 m day−1 and contributed over 95% of the nutrients. At the time-series site where the bay-wide highest Ra activities in the bottom water marked an SGD hotspot with an average rate an order of magnitude greater, the maximum chlorophyll concentration co-occurred, suggesting that SGD may support the algal bloom. The ever-most significant positive correlations between 228Ra and nutrients throughout the water column (P< 0.01, R2 > 0.90 except for soluble reactive phosphorus in the surface) suggested the predominance of SGD in controlling nutrient distribution in the bay. Extrapolated to a national scale, the SGD-carried dissolved inorganic nitrogen flux in small bays was twice as much as those in large bays (>2,000 km2). Thus, the SGD-carried nutrients in small bays merit immediate attention in environmental monitoring and management

Giant third-order nonlinear Hall effect in misfit layer compound (SnS)1.17{1.17}(NbS2_2)3_3

Nonlinear Hall effect (NLHE) holds immense significance in recognizing the band geometry and its potential applications in current rectification. Recent discoveries have expanded the study from second-order to third-order nonlinear Hall effect (THE), which is governed by an intrinsic band geometric quantity called the Berry Connection Polarizability (BCP) tensor. Here we demonstrate a giant THE in a misfit layer compound, (SnS)${1.17}$(NbS$_2$)$_3$. While the THE is prohibited in individual NbS$_2$ and SnS due to the constraints imposed by the crystal symmetry and their band structures, a remarkable THE emerges when a superlattice is formed by introducing a monolayer of SnS. The angular-dependent THE and its scaling relationship indicate that the phenomenon could be correlated to the band geometry modulation, concurrently with the symmetry breaking. The resulting strength of THE is orders of magnitude higher compared to recent studies. Our work illuminates the modulation of structural and electronic geometries for novel quantum phenomena through interface engineering