Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

Darn that dream

Gift of Dr. Mary Jane Esplen.Piano vocal [instrumentation]Love is a strange and powerful thing [first line]Darn that dream [first line of chorus]E flat [key]Slowly [tempo]Popular song [form/genre]Woman on sofa ; Johnny Messner (photograph) [illustration]Publisher's advertisement on inside front and back cover [note

STORYVILLE NIGHTS / Kid Ory, trombone, avec formation de jazz

Comprend : STORYVILLE BLUES / Kid Ory et Johnny St-Cyr - DOCTOR JAZZ / Oliver King et Melrose - MILNEBERG JOYS / Jelly Roll Morton ; Mares et Ropollo - JELLY ROLL BLUES / Jelly Roll Morton - THE WININ' BOY BLUES / Jelly Roll Morton - BOOGABOO / Jelly Roll Morton - SMOKE HOUSE / Luke - DO YOU KNOW WHAT IT MEANS TO MISS NEW-ORLEANS ? / Alter et de LangeBnF-Partenariats, Collection sonore - BelieveContient une table des matière

ON LOVER'S ROAD / ZACHARIAS et ses violons enchantés

Titre uniforme : [Moonglow]Titre uniforme : [Dream]Titre uniforme : [I can't give you anything but love]Titre uniforme : [The man I love]Titre uniforme : [Love is here to stay]Titre uniforme : [Embraceable you]Titre uniforme : [Makin' whoopee!]Titre uniforme : [You go to my head]Comprend : THE MAN I LOVE / George GERSHWIN - BODY AND SOUL / GREEN - MOONGLOW / HUDSON - DE LANGE - MILLS - I'VE GOT YOU UNDER MY SKIN / PORTER - APRIL IN PARIS / DUKE - YOU GOT TO MY HEAD / COOTS - MAKIN' WHOOPEE / DONALDSON - I CAN'T GIVE YOU ANYTHING BUT LOVE / McHUGH - LOVE IS HERE TO STAY / G. GERSHWIN - DREAM / MERCER - EMBRACEABLE YOU / G. GERSHWIN - LOVE FOR SALE / PORTERBnF-Partenariats, Collection sonore - BelieveContient une table des matière

LIONEL PLAYS DRUMS VIBES, PIANO / Lionel Hampton et son orchestre

Titre uniforme : [I know that you know]Titre uniforme : [Darn that dream]Titre uniforme : [Lola = Légende du pays aux oiseaux= Lullaby of birdland]Titre uniforme : [Just one of those things]Titre uniforme : [Star dust]Titre uniforme : [The man I love]Titre uniforme : [Love is here to stay]Comprend : JUST ONE OF THOSE THINGS / Cole Porter - THOUGHTS OF THELMA / Hampton ; Delaney et Frey - THE MAN I LOVE / Gershwin - ONE STEP FROM HEAVEN / Hampton et Frey - DARN THAT DREAM / Delange et van Heusen - STARDUST / Carmichael et Parrish - TRACKING PROBLEM / Hampton et Frey - LULLABY OF BIRDLAND / Shearing - BLUES FOR STEPHEN / Hampton et Frey - AND THE ANGELS SING / Ellman et Mercer - LOVE IS HERE TO STAY / Gershwin - I KNOW THAT YOU KNOW / Harbach ; Caldwell et YoumansBnF-Partenariats, Collection sonore - BelieveContient une table des matière

ERROL PARKER N °4 / Errol PARKER et sa formation

Titre uniforme : [Polka dots and moonbeams]Titre uniforme : [Take the A train]Titre uniforme : [Moonglow]Titre uniforme : [That old devil moon]Titre uniforme : [Smoke gets in your eyes]Titre uniforme : [That old devil moon]Titre uniforme : [Smoke gets in your eyes]Titre uniforme : [Liza]Titre uniforme : [Misty]Comprend : THE BOY NEXT DOOR / H. MARTIN et R. BLANC - Notre concerto / U. BINDI - TAKE THE : A" TRAIN" / Duke ELLINGTON et STRAYHORN - SMOKE GETS IN YOUR EYES / Jerome KERN et HARBACH - SO IN LOVE / Cole PORTER - OLD DEVIL MOON / LANE et HARBURG - POLKA DOTS AND MOONBEAMS / BURKE et Van HEUSEN - THEY SAY THAT FALLING IN LOVE IS WONDERFUL / Irving BERLIN - MISTY / Erroll GARNER - LIZA / George GERSHWIN - MOONGLOW / W. HUSDON - E. DELANGE et I. MILLSBnF-Partenariats, Collection sonore - BelieveContient une table des matière