Feature selection for multi-label learning

Feature Selection plays an important role in machine learning and data mining, and it is often applied as a data pre-processing step. This task can speed up learning algorithms and sometimes improve their performance. In multi-label learning, label dependence is considered another aspect that can contribute to improve learning performance. A replicable and wide systematic review performed by us corroborates this idea. Based on this information, it is believed that considering label dependence during feature selection can lead to better learning performance. The hypothesis of this work is that multi-label feature selection algorithms that consider label dependence will perform better than the ones that disregard it. To this end, we propose multi-label feature selection algorithms that take into account label relations. These algorithms were experimentally compared to the standard approach for feature selection, showing good performance in terms of feature reduction and predictability of the classifiers built using the selected features.São Paulo Research Foundation (FAPESP) (grant 2011/02393-4

A framework to generate synthetic multi-label datasets

A controlled environment based on known properties of the dataset used by a learning algorithm is useful to empirically evaluate machine learning algorithms. Synthetic (artificial) datasets are used for this purpose. Although there are publicly available frameworks to generate synthetic single-label datasets, this is not the case for multi-label datasets, in which each instance is associated with a set of labels usually correlated. This work presents Mldatagen, a multi-label dataset generator framework we have implemented, which is publicly available to the community. Currently, two strategies have been implemented in Mldatagen: hypersphere and hypercube. For each label in the multi-label dataset, these strategies randomly generate a geometric shape (hypersphere or hypercube), which is populated with points (instances) randomly generated. Afterwards, each instance is labeled according to the shapes it belongs to, which defines its multi-label. Experiments with a multi-label classification algorithm in six synthetic datasets illustrate the use of Mldatagen.São Paulo Research Foundation (FAPESP) (grants 2011/02393-4, 2010/15992-0 and 2011/12597-6)Proceedings of the XXXIX Latin American Computing Conference (CLEI 2013).\ud Naiguatá, Venezuela. 7-11 october 2013

Lazy multi-label learning algorithms based on mutuality strategies

Lazy multi-label learning algorithms have become an important research topic within the multi-label community. These algorithms usually consider the set of standard k-Nearest Neighbors of a new instance to predict its labels (multi-label). The prediction is made by following a voting criteria within the multi-labels of the set of k-Nearest Neighbors of the new instance. This work proposes the use of two alternative strategies to identify the set of these examples: the Mutual and Not Mutual Nearest Neighbors rules, which have already been used by lazy single-learning algorithms. In this work, we use these strategies to extend the lazy multi-label algorithm BRkNN. An experimental evaluation carried out to compare both mutuality strategies with the original BRkNN algorithm and the well-known MLkNN lazy algorithm on 15 benchmark datasets showed that MLkNN presented the best predictive performance for the Hamming-Loss evaluation measure, although it was significantly outperformed by the mutuality strategies when F-Measure is considered. The best results of the lazy algorithms were also compared with the results obtained by the Binary Relevance approach using three different base learning algorithms.FAPESP (grants 2010/15992-0, 2011/02393-4, 2011/22749-8 and 2013/12191-5)CNPq (grant 151836/2013-2

Fine-tuning pre-trained neural networks for medical image classification in small clinical datasets

Funding We would like to acknowledge eurekaSD: Enhancing University Research and Education in Areas Useful for Sustainable Development - grants EK14AC0037 and EK15AC0264. We thank Araucária Foundation for the Support of the Scientific and Technological Development of Paraná through a Research and Technological Productivity Scholarship for H. D. Lee (grant 028/2019). We also thank the Brazilian National Council for Scientific and Technological Development (CNPq) through the grant number 142050/2019-9 for A. R. S. Parmezan. The Portuguese team was partially supported by Fundação para a Ciência e a Tecnologia (FCT). R. Fonseca-Pinto was financed by the projects UIDB/50008/2020, UIDP/50008/2020, UIDB/05704/2020 and UIDP/05704/2020 and C. V. Nogueira was financed by the projects UIDB/00013/2020 and UIDP/00013/2020. The funding agencies did not have any further involvement in this paper.Convolutional neural networks have been effective in several applications, arising as a promising supporting tool in a relevant Dermatology problem: skin cancer diagnosis. However, generalizing well can be difficult when little training data is available. The fine-tuning transfer learning strategy has been employed to differentiate properly malignant from non-malignant lesions in dermoscopic images. Fine-tuning a pre-trained network allows one to classify data in the target domain, occasionally with few images, using knowledge acquired in another domain. This work proposes eight fine-tuning settings based on convolutional networks previously trained on ImageNet that can be employed mainly in limited data samples to reduce overfitting risk. They differ on the architecture, the learning rate and the number of unfrozen layer blocks. We evaluated the settings in two public datasets with 104 and 200 dermoscopic images. By finding competitive configurations in small datasets, this paper illustrates that deep learning can be effective if one has only a few dozen malignant and non-malignant lesion images to study and differentiate in Dermatology. The proposal is also flexible and potentially useful for other domains. In fact, it performed satisfactorily in an assessment conducted in a larger dataset with 746 computerized tomographic images associated with the coronavirus disease.info:eu-repo/semantics/publishedVersio

Feature selection for multi-label learning

A presença de atributos não importantes, i.e., atributos irrelevantes ou redundantes nos dados, pode prejudicar o desempenho de classificadores gerados a partir desses dados por algoritmos de aprendizado de máquina. O objetivo de algoritmos de seleção de atributos consiste em identificar esses atributos não importantes para removê-los dos dados antes da construção de classificadores. A seleção de atributos em dados monorrótulo, nos quais cada exemplo do conjunto de treinamento é associado com somente um rótulo, tem sido amplamente estudada na literatura. Entretanto, esse não é o caso para dados multirrótulo, nos quais cada exemplo é associado com um conjunto de rótulos (multirrótulos). Além disso, como esse tipo de dados usualmente apresenta relações entre os rótulos do multirrótulo, algoritmos de aprendizado de máquina deveriam considerar essas relações. De modo similar, a dependência de rótulos deveria também ser explorada por algoritmos de seleção de atributos multirrótulos. A abordagem filtro é uma das mais utilizadas por algoritmos de seleção de atributos, pois ela apresenta um custo computacional potencialmente menor que outras abordagens e utiliza características gerais dos dados para calcular as medidas de importância de atributos. tais como correlação de atributo-classe, entre outras. A hipótese deste trabalho é trabalho é que algoritmos de seleção de atributos em dados multirrótulo que consideram a dependência de rótulos terão um melhor desempenho que aqueles que ignoram essa informação. Para tanto, é proposto como objetivo deste trabalho o projeto e a implementação de algoritmos filtro de seleção de atributos multirrótulo que consideram relações entre rótulos. Em particular, foram propostos dois métodos que levam em conta essas relações por meio da construção de rótulos e da adaptação inovadora do algoritmo de seleção de atributos monorrótulo ReliefF. Esses métodos foram avaliados experimentalmente e apresentam bom desempenho em termos de redução no número de atributos e qualidade dos classificadores construídos usando os atributos selecionados.Irrelevant and/or redundant features in data can deteriorate the performance of the classifiers built from this data by machine learning algorithms. The aim of feature selection algorithms consists in identifying these features and removing them from data before constructing classifiers. Feature selection in single-label data, in which each instance in the training set is associated with only one label, has been widely studied in the literature. However, this is not the case for multi-label data, in which each instance is associated with a set of labels. Moreover, as multi-label data usually exhibit relationships among the labels in the set of labels, machine learning algorithms should take thiis relatinship into account. Therefore, label dependence should also be explored by multi-label feature selection algorithms. The filter approach is one of the most usual approaches considered by feature selection algorithms, as it has potentially lower computational cost than approaches and uses general properties from data to calculate feature importance measures, such as the feature-class correlation. The hypothesis of this work is that feature selection algorithms which consider label dependence will perform better than the ones that disregard label dependence. To this end, ths work proposes and develops filter approach multi-label feature selection algorithms which take into account relations among labels. In particular, we proposed two methods that take into account these relations by performing label construction and adapting the single-label feature selection algorith RelieF. These methods were experimentally evaluated showing good performance in terms of feature reduction and predictability of the classifiers built using the selected features

Evaluating Intelligent Methods for Decision Making Support in Dermoscopy Based on Information Gain and Ensemble

Melanoma, the most dangerous skin cancer, is sometimes associated with a nevus, a relatively common skin lesion. To find early melanoma, nevus, and other lesions, dermoscopy is often used. In this context, intelligent methods have been applied in dermoscopic images to support decision making. A typical computer-aided diagnosis method comprises three steps: (1) extraction of features that describe image properties, (2) selection of important features previously extracted, (3) classification of images based on the selected features. In this work, traditional data mining approaches underexploited in dermoscopy were applied: information gain for feature selection and an ensemble classification method based on gradient boosting. The former technique ranks image features according to data entropy, while the latter combines the outputs of single classifiers to predict the image class. After evaluating these approaches in a public dataset, we can observe that the results obtained are competitive with the state-of-the-art. Moreover, the presented approach allows a reduction of the total number of features and types of features to produce similar classification scores.info:eu-repo/semantics/publishedVersio