Large-Scale Online Semantic Indexing of Biomedical Articles via an Ensemble of Multi-Label Classification Models

Background: In this paper we present the approaches and methods employed in order to deal with a large scale multi-label semantic indexing task of biomedical papers. This work was mainly implemented within the context of the BioASQ challenge of 2014. Methods: The main contribution of this work is a multi-label ensemble method that incorporates a McNemar statistical significance test in order to validate the combination of the constituent machine learning algorithms. Some secondary contributions include a study on the temporal aspects of the BioASQ corpus (observations apply also to the BioASQ's super-set, the PubMed articles collection) and the proper adaptation of the algorithms used to deal with this challenging classification task. Results: The ensemble method we developed is compared to other approaches in experimental scenarios with subsets of the BioASQ corpus giving positive results. During the BioASQ 2014 challenge we obtained the first place during the first batch and the third in the two following batches. Our success in the BioASQ challenge proved that a fully automated machine-learning approach, which does not implement any heuristics and rule-based approaches, can be highly competitive and outperform other approaches in similar challenging contexts

Learning preferences for large scale multi-label problems

Despite that the majority of machine learning approaches aim to solve binary classification problems, several real-world applications require specialized algorithms able to handle many different classes, as in the case of single-label multi-class and multi-label classification problems. The Label Ranking framework is a generalization of the above mentioned settings, which aims to map instances from the input space to a total order over the set of possible labels. However, generally these algorithms are more complex than binary ones, and their application on large-scale datasets could be untractable. The main contribution of this work is the proposal of a novel general online preference-based label ranking framework. The proposed framework is able to solve binary, multi-class, multi-label and ranking problems. A comparison with other baselines has been performed, showing effectiveness and efficiency in a real-world large-scale multi-label task

Deep Neural Networks for Multi-Label Text Classification: Application to Coding Electronic Medical Records

Coding Electronic Medical Records (EMRs) with diagnosis and procedure codes is an essential task for billing, secondary data analyses, and monitoring health trends. Both speed and accuracy of coding are critical. While coding errors could lead to more patient-side financial burden and misinterpretation of a patient’s well-being, timely coding is also needed to avoid backlogs and additional costs for the healthcare facility. Therefore, it is necessary to develop automated diagnosis and procedure code recommendation methods that can be used by professional medical coders. The main difficulty with developing automated EMR coding methods is the nature of the label space. The standardized vocabularies used for medical coding contain over 10 thousand codes. The label space is large, and the label distribution is extremely unbalanced - most codes occur very infrequently, with a few codes occurring several orders of magnitude more than others. A few codes never occur in training dataset at all. In this work, we present three methods to handle the large unbalanced label space. First, we study how to augment EMR training data with biomedical data (research articles indexed on PubMed) to improve the performance of standard neural networks for text classification. PubMed indexes more than 23 million citations. Many of the indexed articles contain relevant information about diagnosis and procedure codes. Therefore, we present a novel method of incorporating this unstructured data in PubMed using transfer learning. Second, we combine ideas from metric learning with recent advances in neural networks to form a novel neural architecture that better handles infrequent codes. And third, we present new methods to predict codes that have never appeared in the training dataset. Overall, our contributions constitute advances in neural multi-label text classification with potential consequences for improving EMR coding

Applying deep learning extreme multi-label classification to the biomedical and multilingual panoramas

Tese de mestrado em Bioinformática e Biologia Computacional, Universidade de Lisboa, Faculdade de Ciências, 2020A indexação automática de documentos é um passo fundamental para a organização de dados e para a extração de informação relevante dos mesmos. Esta extração de informação é realizada através de processos de prospecção de texto e de técnicas de processamento de linguagem natural que tornam a linguagem natural perceptível para o computador. Actualmente, muitas das soluções que são aplicadas a estes processos consistem em soluções de aprendizagem automática. No entanto, tem se assistido a um aumento contínuo da aplicação de soluções de aprendizagem profunda em tarefas de prospecção de texto e de processamento de linguagem natural visto que, graças aos desenvolvimentos contínuos ao longo dos últimos anos, estas soluções têm conseguido obter cada vez melhores resultados. Uma dessas técnicas é a classificação multi-rótulo extrema, uma técnica de processamento de linguagem natural que consiste na indexação de documentos com rótulos pertencentes a um conjunto que pode conter milhares ou mesmo milhões de possíveis rótulos. Este trabalho apresenta um sistema desenvolvido para as ciências biomédicas e para o domínio multilinguístico, através da adaptação de um algoritmo de classificação multi-rótulo extrema usando aprendizagem profunda. O sistema desenvolvido combina ainda um software de reconhecimento de entidades nomeadas com o algoritmo de classificação multi-rótulo extrema de forma a melhorar a atribuição de rótulos aos documentos biomédicos. Para testar o sistema desenvolvido, participei em três competições internacionais com foco na área das ciências biomédicas, nomeadamente na BioASQ task 8a, BioASQ task MESINESP e ainda na subtarefa CODING da competição CANTEMIST. O objectivo comum destas três competições consistia na indexação de documentos biomédicos com rótulos pertencentes a um dado vocabulário biomédico. No entanto, enquanto na task 8a os dados estavam escritos em Inglês, na task MESINESP e na CANTEMIST, os dados biomédicos estavam escritos em Espanhol. Nas competições da BioASQ, o sistema desenvolvido destacou-se sobretudo nas medidas de precisão, superando a grande maioria dos sistemas e ainda alcançando o 1º lugar por duas semanas consecutivas numa das medidas da BioASQ task 8a. Na subtarefa CODING da CANTEMIST, o sistema atingiu uma pontuação de 0.506 na medida mais relevante.Automatic document indexation is a fundamental step for data organization and information retrieval tasks. Information retrieval can be realized through processes of text mining and natural language processing techniques that make natural language understandable to the computer. Nowadays, most solutions that are applied to these processes use machine learning algorithms. However, thanks to continuous developments through recent years, there has been an increasing usage of deep learning solutions applied to text mining and natural language processing tasks, due to the continuous achievement of better results. One of those techniques is extreme multi-label classification, a natural language processing task consisting in the indexation of documents with labels from a label set that may contain thousands or even millions of possible labels. This work presents a system developed for the biomedical and multilingual panoramas based on the adaptation of a deep learning extreme multi-label classification algorithm. The developed system also combines a named entity recognition software with the extreme multi-label classification algorithm in order to improve the label classification of the biomedical documents. To test the developed system, I participated in three international challenges focused on the biomedical sciences, namely in the BioASQ task 8a, BioASQ task MESINESP and in CANTEMIST CODING subtask. The common goal of these three competitions was the indexation of biomedical documents with labels belonging to a specific biomedical vocabulary. However, while the data in task 8a was in English, in task MESINESP and in CANTEMIST the biomedical data was written in Spanish. In the BioASQ competitions, the system stood out in the precision measures, surpassing most competing systems and achieving the 1st place for two consecutive weeks in one evaluation measure in the BioASQ task 8a. In the CANTEMIST CODING subtask, the system achieved a score of 0.506 in the most relevant measure

Results of the BioASQ tasks of the Question Answering Lab at CLEF 2015

International audienceThe goal of the BioASQ challenge is to push research towards highly precise biomedical information access systems. We aim to promote systems and approaches that are able to deal with the whole diversity of the Web, especially for, but not restricted to, the context of bio-medicine. The third challenge consisted of two tasks: semantic indexing and question answering.59 systems by 18 different teams participated in the semantic indexing task (Task 3a).The question answering task was further subdivided into two phases. 24 systems from 9 different teams participates in the annotation phase (Task 3b-phase A), while 26 systems of 10 different teams participated in the answer generation phase (Task 3b-phase B).Overall, the best systems were able to outperform the strong baselines provided by the organizers.In this paper, we present the data used during the challenge as well as the technologies which were used by the participants

Text Classification

There is an abundance of text data in this world but most of it is raw. We need to extract information from this data to make use of it. One way to extract this information from raw text is to apply informative labels drawn from a pre-defined fixed set i.e. Text Classification. In this thesis, we focus on the general problem of text classification, and work towards solving challenges associated to binary/multi-class/multi-label classification. More specifically, we deal with the problem of (i) Zero-shot labels during testing; (ii) Active learning for text screening; (iii) Multi-label classification under low supervision; (iv) Structured label space; (v) Classifying pairs of words in raw text i.e. Relation Extraction. For (i), we use a zero-shot classification model that utilizes independently learned semantic embeddings. Regarding (ii), we propose a novel active learning algorithm that reduces problem of bias in naive active learning algorithms. For (iii), we propose neural candidate-selector architecture that starts from a set of high-recall candidate labels to obtain high-precision predictions. In the case of (iv), we proposed an attention based neural tree decoder that recursively decodes an abstract into the ontology tree. For (v), we propose using second-order relations that are derived by explicitly connecting pairs of words via context token(s) for improved relation extraction. We use a wide variety of both traditional and deep machine learning tools. More specifically, we used traditional machine learning models like multi-valued linear regression and logistic regression for (i, ii), deep convolutional neural networks for (iii), recurrent neural networks for (iv) and transformer networks for (v)