Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency–Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research

Large expert-curated database for benchmarking document similarity detection in biomedical literature search

Document recommendation systems for locating relevant literature have mostly relied on methods developed a decade ago. This is largely due to the lack of a large offline gold-standard benchmark of relevant documents that cover a variety of research fields such that newly developed literature search techniques can be compared, improved and translated into practice. To overcome this bottleneck, we have established the RElevant LIterature SearcH consortium consisting of more than 1500 scientists from 84 countries, who have collectively annotated the relevance of over 180 000 PubMed-listed articles with regard to their respective seed (input) article/s. The majority of annotations were contributed by highly experienced, original authors of the seed articles. The collected data cover 76% of all unique PubMed Medical Subject Headings descriptors. No systematic biases were observed across different experience levels, research fields or time spent on annotations. More importantly, annotations of the same document pairs contributed by different scientists were highly concordant. We further show that the three representative baseline methods used to generate recommended articles for evaluation (Okapi Best Matching 25, Term Frequency-Inverse Document Frequency and PubMed Related Articles) had similar overall performances. Additionally, we found that these methods each tend to produce distinct collections of recommended articles, suggesting that a hybrid method may be required to completely capture all relevant articles. The established database server located at https://relishdb.ict.griffith.edu.au is freely available for the downloading of annotation data and the blind testing of new methods. We expect that this benchmark will be useful for stimulating the development of new powerful techniques for title and title/abstract-based search engines for relevant articles in biomedical research.Peer reviewe

Embeddedness and locational choices: A study of creative workers in a dance organisation

Locational choices of creative workers have been a matter of heated debate over the last decade. This study proposes a micro perspective aimed at disentangling how the individual decision-making process behind locational choices is activated and develops over time. To this aim, we combine previous geographic research on the issue with research on the role of organisational factors in workers\u2019 attraction and retention. Empirically, we carried out an exploratory case study of dancers in a renowned contemporary ballet company based in Reggio Emilia, Italy. With this study, we high- light how matching professional quests and organisation-specific job opportunities activates loca- tional choices, and we extend geographical approaches to embeddedness by considering the role of organisations as crucial mediating entities between the city context and creative workers

Metropolitan and city-regional politics in the urban age: why does “(smart) devolution” matter?

In recent years, two apparently contradictory but, in fact, complementary socio-political phenomena have reinforced each other in the European urban realm: the rescaling of nation-states through “devolution” and the emergence of two opposed versions of “nationalism” (that is, ethnic, non-metropolitanised, state-centric, exclusive, and right-wing populist nationalism and civic, metropolitanized, stateless, inclusive and progressivistemancipatory-social democratic nationalism). In light of these intertwined phenomena, this article shows how an ongoing, pervasive and uneven “metropolitanisation effect” is increasingly shaping city-regional political responses by overlapping metropolitan, cityregional, and national political scales and agendas. This effect is clear in three European cases driven by “civic nationalism” that are altering their referential nation-states’ uniformity through “devolution”. This article compares three metropolitan (and city-regional) cases in the United Kingdom and in Spain, namely, Glasgow (Scotland), Barcelona (Catalonia) and Bilbao (Basque Country), by benchmarking their policy implementation and the tensions produced in reference to their nation-states. Fieldwork was conducted from January 2015 to June 2017 through in-depth interviews with stakeholders in the three locations. Despite the so-called pluri-national and federal dilemmas, this article contributes to the examination of the side effects of “metropolitanisation” by considering three arguments based on geoeconomics (“prosperous competitiveness”), geo-politics (“smart devolution”), and geodemocratics (“right to decide”). Finally, this article adds to the existing research on metropolitan and city-regional politics by demonstrating why “devolution” matters and why it must be considered seriously. The “metropolitanisation effect” is key to understanding and transforming the current configurations of nation-states, such as the United Kingdom and Spain (as we currently know them), beyond internal discord around pluri-nationality and quasi-federalism. This article concludes by suggesting the term “smart devolution” to promote more imaginative and entrepreneurial approaches to metropolitan and city-regional politics, policies, and experimental democracy within these nation-states. These approaches can identify and pursue “smart” avenues of timely, subtle and innovative political strategies for change in the ongoing re-scaling devolution processes occurring in the United Kingdom and in Spain and in the consequent changes in the prospects for the refoundational momentum in the EU