Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

Reporting guideline for the early stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI

A growing number of artificial intelligence (AI)-based clinical decision support systems are showing promising performance in preclinical, in silico, evaluation, but few have yet demonstrated real benefit to patient care. Early stage clinical evaluation is important to assess an AI system’s actual clinical performance at small scale, ensure its safety, evaluate the human factors surrounding its use, and pave the way to further large scale trials. However, the reporting of these early studies remains inadequate. The present statement provides a multistakeholder, consensus-based reporting guideline for the Developmental and Exploratory Clinical Investigations of DEcision support systems driven by Artificial Intelligence (DECIDE-AI). We conducted a two round, modified Delphi process to collect and analyse expert opinion on the reporting of early clinical evaluation of AI systems. Experts were recruited from 20 predefined stakeholder categories. The final composition and wording of the guideline was determined at a virtual consensus meeting. The checklist and the Explanation & Elaboration (E&E) sections were refined based on feedback from a qualitative evaluation process. 123 experts participated in the first round of Delphi, 138 in the second, 16 in the consensus meeting, and 16 in the qualitative evaluation. The DECIDE-AI reporting guideline comprises 17 AI specific reporting items (made of 28 subitems) and 10 generic reporting items, with an E&E paragraph provided for each. Through consultation and consensus with a range of stakeholders, we have developed a guideline comprising key items that should be reported in early stage clinical studies of AI-based decision support systems in healthcare. By providing an actionable checklist of minimal reporting items, the DECIDE-AI guideline will facilitate the appraisal of these studies and replicability of their findings

Search for heavy resonances decaying into a Z or W boson and a Higgs boson in final states with leptons and b-jets in 139 fb−1 of pp collisions at s√ = 13 TeV with the ATLAS detector

This article presents a search for new resonances decaying into a Z or W boson and a 125 GeV Higgs boson h, and it targets the νν¯¯¯bb¯¯, ℓ+ℓ−bb¯¯, or ℓ±νbb¯¯ final states, where ℓ = e or μ, in proton-proton collisions at s√ = 13 TeV. The data used correspond to a total integrated luminosity of 139 fb−1 collected by the ATLAS detector during Run 2 of the LHC at CERN. The search is conducted by examining the reconstructed invariant or transverse mass distributions of Zh or Wh candidates for evidence of a localised excess in the mass range from 220 GeV to 5 TeV. No significant excess is observed and 95% confidence-level upper limits between 1.3 pb and 0.3 fb are placed on the production cross section times branching fraction of neutral and charged spin-1 resonances and CP-odd scalar bosons. These limits are converted into constraints on the parameter space of the Heavy Vector Triplet model and the two-Higgs-doublet model

Search for boosted diphoton resonances in the 10 to 70 GeV mass range using 138 fb−1 of 13 TeV pp collisions with the ATLAS detector

A search for diphoton resonances in the mass range between 10 and 70 GeV with the ATLAS experiment at the Large Hadron Collider (LHC) is presented. The analysis is based on pp collision data corresponding to an integrated luminosity of 138 fb−1 at a centre-of-mass energy of 13 TeV recorded from 2015 to 2018. Previous searches for diphoton resonances at the LHC have explored masses down to 65 GeV, finding no evidence of new particles. This search exploits the particular kinematics of events with pairs of closely spaced photons reconstructed in the detector, allowing examination of invariant masses down to 10 GeV. The presented strategy covers a region previously unexplored at hadron colliders because of the experimental challenges of recording low-energy photons and estimating the backgrounds. No significant excess is observed and the reported limits provide the strongest bound on promptly decaying axion-like particles coupling to gluons and photons for masses between 10 and 70 GeV

Search for flavour-changing neutral current interactions of the top quark and the Higgs boson in events with a pair of τ-leptons in pp collisions at s√ = 13 TeV with the ATLAS detector

A search for flavour-changing neutral current (FCNC) tqH interactions involving a top quark, another up-type quark (q = u, c), and a Standard Model (SM) Higgs boson decaying into a τ-lepton pair (H → τ+τ−) is presented. The search is based on a dataset of pp collisions at s√ = 13 TeV that corresponds to an integrated luminosity of 139 fb−1 recorded with the ATLAS detector at the Large Hadron Collider. Two processes are considered: single top quark FCNC production in association with a Higgs boson (pp → tH), and top quark pair production in which one of top quarks decays into Wb and the other decays into qH through the FCNC interactions. The search selects events with two hadronically decaying τ-lepton candidates (τhad) or at least one τhad with an additional lepton (e, μ), as well as multiple jets. Event kinematics is used to separate signal from the background through a multivariate discriminant. A slight excess of data is observed with a significance of 2.3σ above the expected SM background, and 95% CL upper limits on the t → qH branching ratios are derived. The observed (expected) 95% CL upper limits set on the t → cH and t → uH branching ratios are 9.4×10−4(4.8+2.2−1.4×10−4) and 6.9×10−4(3.5+1.5−1.0×10−4), respectively. The corresponding combined observed (expected) upper limits on the dimension-6 operator Wilson coefficients in the effective tqH couplings are Ccϕ < 1.35 (0.97) and Cuϕ < 1.16 (0.82)

Evidence for the charge asymmetry in pp → tt¯ production at s√ = 13 TeV with the ATLAS detector

Inclusive and differential measurements of the top–antitop (tt¯) charge asymmetry Att¯C and the leptonic asymmetry Aℓℓ¯C are presented in proton–proton collisions at s√ = 13 TeV recorded by the ATLAS experiment at the CERN Large Hadron Collider. The measurement uses the complete Run 2 dataset, corresponding to an integrated luminosity of 139 fb−1, combines data in the single-lepton and dilepton channels, and employs reconstruction techniques adapted to both the resolved and boosted topologies. A Bayesian unfolding procedure is performed to correct for detector resolution and acceptance effects. The combined inclusive tt¯ charge asymmetry is measured to be Att¯C = 0.0068 ± 0.0015, which differs from zero by 4.7 standard deviations. Differential measurements are performed as a function of the invariant mass, transverse momentum and longitudinal boost of the tt¯ system. Both the inclusive and differential measurements are found to be compatible with the Standard Model predictions, at next-to-next-to-leading order in quantum chromodynamics perturbation theory with next-to-leading-order electroweak corrections. The measurements are interpreted in the framework of the Standard Model effective field theory, placing competitive bounds on several Wilson coefficients

Measurements of Zγ+jets differential cross sections in pp collisions at s√ = 13 TeV with the ATLAS detector

Differential cross-section measurements of Zγ production in association with hadronic jets are presented, using the full 139 fb−1 dataset of s√ = 13 TeV proton–proton collisions collected by the ATLAS detector during Run 2 of the LHC. Distributions are measured using events in which the Z boson decays leptonically and the photon is usually radiated from an initial-state quark. Measurements are made in both one and two observables, including those sensitive to the hard scattering in the event and others which probe additional soft and collinear radiation. Different Standard Model predictions, from both parton-shower Monte Carlo simulation and fixed-order QCD calculations, are compared with the measurements. In general, good agreement is observed between data and predictions from MATRIX and MiNNLOPS, as well as next-to-leading-order predictions from MADGRAPH5_AMC@NLO and SHERPA

Search for the Zγ decay mode of new high-mass resonances in pp collisions at √s = 13 TeV with the ATLAS detector

This letter presents a search for narrow, high-mass resonances in the Zγ final state with the Z boson decaying into a pair of electrons or muons. The √s = 13 TeV pp collision data were recorded by the ATLAS detector at the CERN Large Hadron Collider and have an integrated luminosity of 140 fb−1. The data are found to be in agreement with the Standard Model background expectation. Upper limits are set on the resonance production cross section times the decay branching ratio into Zγ. For spin-0 resonances produced via gluon–gluon fusion, the observed limits at 95% confidence level vary between 65.5 fb and 0.6 fb, while for spin-2 resonances produced via gluon–gluon fusion (or quark–antiquark initial states) limits vary between 77.4 (76.1) fb and 0.6 (0.5) fb, for the mass range from 220 GeV to 3400 GeV

Search for lepton-favour violation in high-mass dilepton final states using 139 fb−1 of pp collisions at √s = 13 TeV with the ATLAS detector

A search is performed for a heavy particle decaying into different-flavour, dilepton final states, using 139 fb−1 of proton-proton collision data at √s = 13 TeV collected in 2015–2018 by the ATLAS detector at the Large Hadron Collider. Final states with electrons, muons and hadronically decaying tau leptons are considered (eμ, eτ or μτ). No significant excess over the Standard Model predictions is observed. Upper limits on the production cross-section are set as a function of the mass of a Z′ boson, a supersymmetric τ-sneutrino, and a quantum black-hole. The observed 95% CL lower mass limits obtained on a typical benchmark model Z′ boson are 5.0 TeV (eμ), 4.0 TeV (eτ), and 3.9 TeV (μτ), respectively