Early stopping by correlating online indicators in neural networks

Financiado para publicación en acceso aberto: Universidade de Vigo/CISUGinfo:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2013-2016/TIN2017-85160-C2-2-R/ES/AVANCES EN NUEVOS SISTEMAS DE EXTRACCION DE RESPUESTAS CON ANALISIS SEMANTICO Y APRENDIZAJE PROFUNDOinfo:eu-repo/grantAgreement/AEI/Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020/PID2020-113230RB-C22/ES/SEQUENCE LABELING MULTITASK MODELS FOR LINGUISTICALLY ENRICHED NER: SEMANTICS AND DOMAIN ADAPTATION (SCANNER-UVIGO)In order to minimize the generalization error in neural networks, a novel technique to identify overfitting phenomena when training the learner is formally introduced. This enables support of a reliable and trustworthy early stopping condition, thus improving the predictive power of that type of modeling. Our proposal exploits the correlation over time in a collection of online indicators, namely characteristic functions for indicating if a set of hypotheses are met, associated with a range of independent stopping conditions built from a canary judgment to evaluate the presence of overfitting. That way, we provide a formal basis for decision making in terms of interrupting the learning process. As opposed to previous approaches focused on a single criterion, we take advantage of subsidiarities between independent assessments, thus seeking both a wider operating range and greater diagnostic reliability. With a view to illustrating the effectiveness of the halting condition described, we choose to work in the sphere of natural language processing, an operational continuum increasingly based on machine learning. As a case study, we focus on parser generation, one of the most demanding and complex tasks in the domain. The selection of cross-validation as a canary function enables an actual comparison with the most representative early stopping conditions based on overfitting identification, pointing to a promising start toward an optimal bias and variance control.Agencia Estatal de Investigación | Ref. TIN2017-85160-C2-2-RAgencia Estatal de Investigación | Ref. PID2020-113230RB-C22Xunta de Galicia | Ref. ED431C 2018/5

A Study of Dynamic Populations in Geometric Semantic Genetic Programming

Farinati, D., Bakurov, I., & Vanneschi, L. (2023). A Study of Dynamic Populations in Geometric Semantic Genetic Programming. Information Sciences, 648(November), 1-21. [119513]. https://doi.org/10.1016/j.ins.2023.119513 --- This work was supported by national funds through FCT (Fundação para a Ciência e a Tecnologia), under the project - UIDB/04152/2020 - Centro de Investigação em Gestão de Informação (MagIC)/NOVA IMS.Allowing the population size to variate during the evolution can bring advantages to evolutionary algorithms (EAs), retaining computational effort during the evolution process. Dynamic populations use computational resources wisely in several types of EAs, including genetic programming. However, so far, a thorough study on the use of dynamic populations in Geometric Semantic Genetic Programming (GSGP) is missing. Still, GSGP is a resource-greedy algorithm, and the use of dynamic populations seems appropriate. This paper adapts algorithms to GSGP to manage dynamic populations that were successful for other types of EAs and introduces two novel algorithms. The novel algorithms exploit the concept of semantic neighbourhood. These methods are assessed and compared through a set of eight regression problems. The results indicate that the algorithms outperform standard GSGP, confirming the suitability of dynamic populations for GSGP. Interestingly, the novel algorithms that use semantic neighbourhood to manage variation in population size are particularly effective in generating robust models even for the most difficult of the studied test problems.publishersversionpublishe

Hyperparameters optimization on neural networks for bond trading

Project Work presented as the partial requirement for obtaining a Master's degree in Statistics and Information Management, specialization in Risk Analysis and ManagementArtificial Neural Networks have been recently spotlighted as de facto tools used for classification. Their ability to deal with complex decision boundaries makes them potentially suitable to work on trading within financial markets, namely on Bonds. Such classifier faces high flexibility on its parameters in parallel with great modularity of its techniques, arising thus the need to efficiently optimize its hyperparameters. To determine the most effcient search method to optimize almost the majority of the Neural Networks hyperparameters, we have compared the results obtained by the manual, evolutionary (genetic algorithm) and random search methods. The search methods compete on several metrics from which we aim to estimate the generalization capability, i.e. the capacity to correctly predict on unseen data. We have found the manual method to present better generalization results than the remaining automatic methods. Also, no benefit was found on the direction provided by the genetic search method when compared to the purely random. Such results demonstrate the importance of human oversight during the hyperparameters optimization and weight training phases, capable of analyzing in parallel multiple metrics and data visualization techniques, a process critical to avoid suboptimal solutions when navigating complex hyperspaces

Can machine learning methods contribute as a decision support system in sequential oligometastatic radioablation therapy?

Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsCancer treatment is among the major medical challenges of this century. Sequential oligometastatic radio-ablation (SOMA) is a novel treatment method that aims at ablating reoccurring metastasis in a single session with a targeted high dose of radiation. To know if SOMA is the best possible treatment method for a patient, the benefits of each available therapy need to be understood and evaluated. The ability to model complex systems, such as cancer treatment, is the strength of machine learning techniques. These techniques have improved the understanding of numerous medical therapies already. In some cases, they can serve as medical support systems if they deliver reliable results that doctors can trust and understand. The results obtained from applying numerous machine learning techniques to the data of SOMA-treated patients show that there are favorable techniques in some cases. It was observed that the Random Forest algorithm proved superior at different classification tasks. Additionally, regression problems opposed a great challenge, as the amount of data is very limited. Finally, SHAP values - a novel machine learning interpretation technique – provided valuable insights into understanding the rationale of each algorithm. They proved that the machine learning algorithms could learn patterns aligned with the human intuition in the problems presented. SHAP values show great potential in bridging the gap between complex machine learning algorithms and their interpretability. They display how an algorithm learns from the data and derives results. This opens up exciting possibilities for applying machine learning algorithms in the real world

Geometric semantic inspired mutation for M3GP

Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsOne of the most challenging Machine Learning tasks is multiclass classification. Genetic Programming (GP) is not able to achieve a very good performance when applied to classification problems with number of classes bigger than two. However, Multidimensional Multiclass Genetic Programming (M2GP) and Multidimensional Multiclass Genetic Programming with Multidimensional Populations (M3GP), two wrapper-based GP classifiers, have shown to be competitive with state-of-the-art classifiers. The main focus of this work is a new version of M3GP, called Geometric Semantic In- spired M3GP (GSI-M3GP), inspired in geometric semantic operators. GSI-M3GP works in the same way as M3GP, but uses only three operators to create new individuals: add branch, remove branch and a new mutation operator called geometric semantic inspired mutation (gsimutation). In order to test GSI-M3GP and compare it to M3GP, an implementation in Java was developed. Nine different versions of GSI-M3GP were created and tested on eight benchmark problems. For most of the versions of GSI-M3GP, the new algorithm is competitive with M3GP on all the problems. Additionally, it was tested if adding a crossover operator would improve the results, which it did not. A few other alterations were made to the original M3GP algorithm to test the possibility of using the Euclidean distance, instead of the Mahalanobis distance, without harming the quality of the solutions. These alterations do not always maintain the quality of the solutions.Uma das tarefas mais desafiantes de Aprendizagem Automática é classificação em mais de duas classes. Genetic Programming (GP) não consegue obter um bom desempenho nestes problemas. No entanto, Multidimensional Multiclass Genetic Programming (M2GP) e Multi-dimensional Multi class Genetic Programming with Multidimensional Populations (M3GP), dois algoritmos de classificação que utilizam GP como método wrapper, mostraram ser competitivos com classificadores do estado-de-arte. O foco deste trabalho e a criação de uma nova versão de M3GP, chamada Geometric Semantic Inspired M3GP (GSI-M3GP), inspirada em operadores da geometria semântica. GSI-M3GP funciona da mesma forma que M3GP, mas utiliza apenas três operadores para criar novos indivídulos: adicionar dimensão, remover dimensão e um novo operador de mutação, de nome geometric semantic inspired mutation (gsi-mutation). Para testar GSI-M3GP e comparámo-lo com M3GP, foi criada uma implementação em Java. Foram testadas nove versões diferentes de GSI-M3GP em oito problemas de benchmark. GSI- M3GP _e competitivo com M3GP em todos os problemas considerados. Foi ainda testado se adicionar um operador de crossover melhoraria os resultados, mas tal não se verificou. Outras alterações foram feitas a M3GP de forma a testar a possibilidade de utilizar a distância Euclideana em vez da distância de Mahalanobis, sem que a qualidade das soluções fosse afetada. Estas alterações nem sempre mantêm a qualidade das soluções

Analyse prédictive par inférence de structures à partir de dossiers électroniques de patients

International audienc

A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

Classification of EEG data using machine learning techniques

Automatic interpretation of reading from the brain could allow for many interesting applications including movement of prosthetic limbs and more seamless manmachine interaction. This work studied classification of EEG signals used in a study of memory. The goal was to evaluate the performance of the state of the art algorithms. A secondary goal was to try to improve upon the result of a method that was used in a study similar to the one used in this work. For the experiment, the signals were transformed into the frequency domain and their magnitudes were used as features. A subset of these features was then selected and fed into a support vector machine classifier. The first part of this work tried to improve the selection of features that was used to discriminate between different memory categories. The second part investigated the uses of time series as features instead of time points. Two feature selection methods, genetic algorithm and correlation-based, were implemented and tested. Both of them performed worse than the baseline ANOVA method. The time series classifier also performed worse than the standard classifier. However, experiments showed that there was information to gain by using the time series, motivating more advanced methods to be explored. Both the results achieved by this thesis and in other work are above chance. However, high accuracies can only be achieved at the cost of long delays and few output alternatives. This limits the information that can be extracted from the EEG sensor and its usability