Validation of automatic similarity measures

Tese de mestrado em Bioinformática e Biologia Computacional, Universidade de Lisboa, Faculdade de Ciências, 2020A capacidade para comparar automaticamente duas entidades biomédicas (p. ex. doenças, vias metabólicas ou artigos científicos) permite que os computadores raciocinem sobre o conhecimento científico. Assim sendo, fazer a validação destas medidas é essencial para garantir que os resultados produzidos por elas reflictam o actual conhecimento colectivo sobre o respectivo domínio. Uma das estratégias para avaliar se a medida é precisa e funcional é a validação manual por parte de peritos. Contudo, este processo é ineficiente devido a toda a pesquisa secundária necessária para o fazer, o que significa que compilar grandes conjuntos de dados de valores de semelhança atribuídos por humanos é uma tarefa difícil. “Manual Validation Helper Tool” (MVHT) é uma aplicação web criada com o intuito de acelerar esta validação manual, em conjunto com um formato capaz de acomodar os diversos tipos de dados em forma de anotações, provenientes de diferentes ontologias ou domínios. MVHT foi testada em quatro datasets distintos e um deles foi apresentado a utilizadores piloto para que dessem o seu feedback acerca do que poderia ser melhorado na aplicação, bem como para se obter um gold-standard de semelhança manual. Com o seu auxílio, a ferramenta foi optimizada e encontra-se acessível para ser usada por criadores de medidas de semelhança semântica, que por sua vez podem partilhar os seus datasets de forma prática, os quais peritos podem visitar e rapidamente começar a comparar pares de entidades.The ability to automatically compare two biomedical entities (e.g. diseases, biochemical pathways, papers) enables the use of computers to reason over scientific knowledge. As such, validating these measures is essential to ensure that the results they produce reflect the current community knowledge on the respective domain. Manual validation by experts is one of the strategies to assess whether a measure is sound and accurate. However, this is an inefficient process because of the secondary research required to do so, which means that compiling large datasets of human-curated similarity values is difficult. The “Manual Validation Helper Tool” (MVHT) is a web application created to accelerate this manual validation, coupled to a format that can accommodate different types of data in the form of annotations, from different domains or ontologies. MVHT was tested on four distinct datasets and one of them was given to pilot users so they could provide feedback on the application, as well as to gather a gold-standard of manual similarity. With their help the tool was optimized and is accessible to be used by creators of semantic similarity measures, who can share their datasets in a more practical way via generated URLs, which other people can visit and quickly start comparing pairs of entities

Identifying Relevant Evidence for Systematic Reviews and Review Updates

Systematic reviews identify, assess and synthesise the evidence available to answer complex research questions. They are essential in healthcare, where the volume of evidence in scientific research publications is vast and cannot feasibly be identified or analysed by individual clinicians or decision makers. However, the process of creating a systematic review is time consuming and expensive. The pace of scientific publication in medicine and related fields also means that evidence bases are continually changing and review conclusions can quickly become out of date. Therefore, developing methods to support the creating and updating of reviews is essential to reduce the workload required and thereby ensure that reviews remain up to date. This research aims to support systematic reviews, thus improving healthcare through natural language processing and information retrieval techniques. More specifically, this thesis aims to support the process of identifying relevant evidence for systematic reviews and review updates to reduce the workload required from researchers. This research proposes methods to improve studies ranking for systematic reviews. In addition, this thesis describes a dataset of systematic review updates in the field of medicine created using 25 Cochrane reviews. Moreover, this thesis develops an algorithm to automatically refine the Boolean query to improve the identification of relevant studies for review updates. The research demonstrates that automating the process of identifying relevant evidence can reduce the workload of conducting and updating systematic reviews