Bioinspired Architecture Selection for Multitask Learning

Faced with a new concept to learn, our brain does not work in isolation. It uses all previously learned knowledge. In addition, the brain is able to isolate the knowledge that does not benefit us, and to use what is actually useful. In machine learning, we do not usually benefit from the knowledge of other learned tasks. However, there is a methodology called Multitask Learning (MTL), which is based on the idea that learning a task along with other related tasks produces a transfer of information between them, what can be advantageous for learning the first one. This paper presents a new method to completely design MTL architectures, by including the selection of the most helpful subtasks for the learning of the main task, and the optimal network connections. In this sense, the proposed method realizes a complete design of the MTL schemes. The method is simple and uses the advantages of the Extreme Learning Machine to automatically design a MTL machine, eliminating those factors that hinder, or do not benefit, the learning process of the main task. This architecture is unique and it is obtained without testing/error methodologies that increase the computational complexity. The results obtained over several real problems show the good performances of the designed networks with this method

Learn, don't forget: constructive methods for effective continual learning

L'objectiu distintiu de la intel·ligència artificial és aconseguir agents amb capacitat per adaptar-se a fluxos continus d'informació. L'aprenentatge continu pretén donar resposta a aquest repte. No obstant això, els models d'aprenentatge automàtic acumulen el coneixement d'una manera diferent de la dels humans, i l'aprenentatge de noves tasques condueix a la degradació de les passades, fenomen anomenat "oblit catastròfic". La majoria dels mètodes d'aprenentatge continu o penalitzen el canvi dels paràmetres considerats importants per a les tasques passades (mètodes basats en la regularització) o bé emprenen una petita memòria intermèdia de repetició (mètodes basats en la repetició) que alimenta el model amb exemples de tasques passades per preservar el rendiment. Tot i això, el paper exacte que juga la regularització i els altres possibles factors que fan que el procés d'aprenentatge continu sigui eficaç no es coneixen bé. El projecte dóna llum sobre aquestes qüestions i suggereix maneres de millorar el rendiment de l'aprenentatge continu en tasques de visió com la classificació.El objetivo distintivo de la inteligencia artificial reside en conseguir agentes con capacidad para adaptarse a flujos continuos de información. El aprendizaje continuo pretende dar respuesta a este reto. Sin embargo, los modelos de aprendizaje automático acumulan el conocimiento de una manera diferente a la de los humanos, y el aprendizaje de nuevas tareas conduce a la degradación de las pasadas, fenómeno denominado "olvido catastrófico". La mayoría de los métodos de aprendizaje continuo o bien penalizan el cambio de los parámetros considerados importantes para las tareas pasadas (métodos basados en la regularización) o bien emplean un pequeño búfer de repetición (métodos basados en la repetición) que alimenta el modelo con ejemplos de tareas pasadas para preservar el rendimiento. Sin embargo, el papel exacto que juega la regularización y los demás posibles factores que hacen que el proceso de aprendizaje continuo sea eficaz no se conocen bien. El proyecto arroja luz sobre estas cuestiones y sugiere formas de mejorar el rendimiento del aprendizaje continuo en tareas de visión como la clasificación.The hallmark of artificial intelligence lies in agents with capabilities to adapt to continuous streams of information and tasks. Continual Learning aims to address this challenge. However, machine learning models accumulate knowledge in a manner different from humans, and learning new tasks leads to degradation in past ones, a phenomenon aptly named "catastrophic forgetting". Most continual learning methods either penalize the change of parameters deemed important for past tasks (regularization-based methods) or employ a small replay buffer (replay-based methods) that feeds the model examples from past tasks in order to preserve performance. However, the role and nature of the regularization and the other possible factors that make the continual learning process effective are not well understood. The project sheds light on these questions and suggests ways to improve the performance of continual learning in vision tasks such as classification.Outgoin

Deep Artificial Neural Networks and Neuromorphic Chips for Big Data Analysis: Pharmaceutical and Bioinformatics Applications

[Abstract] Over the past decade, Deep Artificial Neural Networks (DNNs) have become the state-of-the-art algorithms in Machine Learning (ML), speech recognition, computer vision, natural language processing and many other tasks. This was made possible by the advancement in Big Data, Deep Learning (DL) and drastically increased chip processing abilities, especially general-purpose graphical processing units (GPGPUs). All this has created a growing interest in making the most of the potential offered by DNNs in almost every field. An overview of the main architectures of DNNs, and their usefulness in Pharmacology and Bioinformatics are presented in this work. The featured applications are: drug design, virtual screening (VS), Quantitative Structure–Activity Relationship (QSAR) research, protein structure prediction and genomics (and other omics) data mining. The future need of neuromorphic hardware for DNNs is also discussed, and the two most advanced chips are reviewed: IBM TrueNorth and SpiNNaker. In addition, this review points out the importance of considering not only neurons, as DNNs and neuromorphic chips should also include glial cells, given the proven importance of astrocytes, a type of glial cell which contributes to information processing in the brain. The Deep Artificial Neuron–Astrocyte Networks (DANAN) could overcome the difficulties in architecture design, learning process and scalability of the current ML methods.Galicia. Consellería de Cultura, Educación e Ordenación Universitaria; GRC2014/049Galicia. Consellería de Cultura, Educación e Ordenación Universitaria; R2014/039Instituto de Salud Carlos III; PI13/0028

Bio-inspired computation for big data fusion, storage, processing, learning and visualization: state of the art and future directions

This overview gravitates on research achievements that have recently emerged from the confluence between Big Data technologies and bio-inspired computation. A manifold of reasons can be identified for the profitable synergy between these two paradigms, all rooted on the adaptability, intelligence and robustness that biologically inspired principles can provide to technologies aimed to manage, retrieve, fuse and process Big Data efficiently. We delve into this research field by first analyzing in depth the existing literature, with a focus on advances reported in the last few years. This prior literature analysis is complemented by an identification of the new trends and open challenges in Big Data that remain unsolved to date, and that can be effectively addressed by bio-inspired algorithms. As a second contribution, this work elaborates on how bio-inspired algorithms need to be adapted for their use in a Big Data context, in which data fusion becomes crucial as a previous step to allow processing and mining several and potentially heterogeneous data sources. This analysis allows exploring and comparing the scope and efficiency of existing approaches across different problems and domains, with the purpose of identifying new potential applications and research niches. Finally, this survey highlights open issues that remain unsolved to date in this research avenue, alongside a prescription of recommendations for future research.This work has received funding support from the Basque Government (Eusko Jaurlaritza) through the Consolidated Research Group MATHMODE (IT1294-19), EMAITEK and ELK ARTEK programs. D. Camacho also acknowledges support from the Spanish Ministry of Science and Education under PID2020-117263GB-100 grant (FightDIS), the Comunidad Autonoma de Madrid under S2018/TCS-4566 grant (CYNAMON), and the CHIST ERA 2017 BDSI PACMEL Project (PCI2019-103623, Spain)

Efficient Learning Machines

Computer scienc

Artificial Olfaction in the 21st Century

The human olfactory system remains one of the most challenging biological systems to replicate. Humans use it without thinking, where it can measure offer protection from harm and bring enjoyment in equal measure. It is the system's real-time ability to detect and analyze complex odors that makes it difficult to replicate. The field of artificial olfaction has recruited and stimulated interdisciplinary research and commercial development for several applications that include malodor measurement, medical diagnostics, food and beverage quality, environment and security. Over the last century, innovative engineers and scientists have been focused on solving a range of problems associated with measurement and control of odor. The IEEE Sensors Journal has published Special Issues on olfaction in 2002 and 2012. Here we continue that coverage. In this article, we summarize early work in the 20th Century that served as the foundation upon which we have been building our odor-monitoring instrumental and measurement systems. We then examine the current state of the art that has been achieved over the last two decades as we have transitioned into the 21st Century. Much has been accomplished, but great progress is needed in sensor technology, system design, product manufacture and performance standards. In the final section, we predict levels of performance and ubiquitous applications that will be realized during in the mid to late 21st Century

Quantum Biomimetics

136 p.En esta tesis proponemos el concepto de Biomimética Cuántica orientado hacia la reproducción de comportamientos propios de los seres vivos en protocolos de información cuántica. En concreto, las propiedades que aspiramos a imitar emergen como resultado de fenómenos de interacción en diferentes escalas, resultando inaccesibles para un tratamiento matemático acorde al ofrecido por las plataformas de tecnologías cuánticas. Por tanto, el objetivo de la tesis es el de diseñar modelos con cabida para las mencionadas características biológicas pero simplificados de forma que puedan ser adaptados en protocolos experimentales. La tesis se divide en tres partes, una por cada rasgo biológico diferente empleado como inspiración: selección natural, memoria e inteligencia. El estudio presentado en la primera parte culmina con la obtención de un modelo de vida artificial con una identidad exclusivamente cuántica, que no solo permite la escenificación del modelo de selección natural a escala microscópica si no que proporciona un posible marco para la implementación de algoritmos genéticos y problemas de optimización en plataformas cuánticas. En la segunda parte se muestran algoritmos asociados con la simulación de evolución temporal regida por ecuaciones con una dependencia explicita en términos deslocalizados temporalmente. Estos permiten la incorporación de la retroalimentación y posalimentación al conjunto de herramientas en información cuántica. La tercera y última parte versa acerca de la posible simbiosis entre los algoritmos de aprendizaje y los protocolos cuánticos. Mostramos como aplicar técnicas de optimización clásicas para tratar problemas cuánticos así como la codificación y resolución de problemas en dinámicas puramente cuánticas

Functions and mechanisms of intrinsic motivations: the knowledge versus competence distinction

Mammals, and humans in particular, are endowed with an exceptional capacity for cumulative learning. This capacity crucially de- pends on the presence of intrinsic motivations, i.e. motivations that are not directly related to an organism\u27s survival and reproduction but rather to its ability to learn. Recently, there have been a number of attempts to model and reproduce intrinsic motivations in artificial systems. Different kinds of intrinsic motivations have been proposed both in psychology and in machine learning and robotics: some are based on the knowl- edge of the learning system, while others are based on its competence. In this contribution we discuss the distinction between knowledge-based and competence-based intrinsic motivations with respect to both the functional roles that motivations play in learning and the mechanisms by which those functions are implemented. In particular, after arguing that the principal function of intrinsic motivations consists in allowing the development of a repertoire of skills (rather than of knowledge), we suggest that at least two different sub-functions can be identified: (a) discovering which skills might be acquired and (b) deciding which skill to train when. We propose that in biological organisms knowledge-based intrinsic motivation mechanisms might implement the former function, whereas competence-based mechanisms might underly the latter one

Agrupamiento, predicción y clasificación ordinal para series temporales utilizando técnicas de machine learning: aplicaciones

In the last years, there has been an increase in the number of fields improving their standard processes by using machine learning (ML) techniques. The main reason for this is that the vast amount of data generated by these processes is difficult to be processed by humans. Therefore, the development of automatic methods to process and extract relevant information from these data processes is of great necessity, giving that these approaches could lead to an increase in the economic benefit of enterprises or to a reduction in the workload of some current employments. Concretely, in this Thesis, ML approaches are applied to problems concerning time series data. Time series is a special kind of data in which data points are collected chronologically. Time series are present in a wide variety of fields, such as atmospheric events or engineering applications. Besides, according to the main objective to be satisfied, there are different tasks in the literature applied to time series. Some of them are those on which this Thesis is mainly focused: clustering, classification, prediction and, in general, analysis. Generally, the amount of data to be processed is huge, arising the need of methods able to reduce the dimensionality of time series without decreasing the amount of information. In this sense, the application of time series segmentation procedures dividing the time series into different subsequences is a good option, given that each segment defines a specific behaviour. Once the different segments are obtained, the use of statistical features to characterise them is an excellent way to maximise the information of the time series and simultaneously reducing considerably their dimensionality. In the case of time series clustering, the objective is to find groups of similar time series with the idea of discovering interesting patterns in time series datasets. In this Thesis, we have developed a novel time series clustering technique. The aim of this proposal is twofold: to reduce as much as possible the dimensionality and to develop a time series clustering approach able to outperform current state-of-the-art techniques. In this sense, for the first objective, the time series are segmented in order to divide the them identifying different behaviours. Then, these segments are projected into a vector of statistical features aiming to reduce the dimensionality of the time series. Once this preprocessing step is done, the clustering of the time series is carried out, with a significantly lower computational load. This novel approach has been tested on all the time series datasets available in the University of East Anglia and University of California Riverside (UEA/UCR) time series classification (TSC) repository. Regarding time series classification, two main paths could be differentiated: firstly, nominal TSC, which is a well-known field involving a wide variety of proposals and transformations applied to time series. Concretely, one of the most popular transformation is the shapelet transform (ST), which has been widely used in this field. The original method extracts shapelets from the original time series and uses them for classification purposes. Nevertheless, the full enumeration of all possible shapelets is very time consuming. Therefore, in this Thesis, we have developed a hybrid method that starts with the best shapelets extracted by using the original approach with a time constraint and then tunes these shapelets by using a convolutional neural network (CNN) model. Secondly, time series ordinal classification (TSOC) is an unexplored field beginning with this Thesis. In this way, we have adapted the original ST to the ordinal classification (OC) paradigm by proposing several shapelet quality measures taking advantage of the ordinal information of the time series. This methodology leads to better results than the state-of-the-art TSC techniques for those ordinal time series datasets. All these proposals have been tested on all the time series datasets available in the UEA/UCR TSC repository. With respect to time series prediction, it is based on estimating the next value or values of the time series by considering the previous ones. In this Thesis, several different approaches have been considered depending on the problem to be solved. Firstly, the prediction of low-visibility events produced by fog conditions is carried out by means of hybrid autoregressive models (ARs) combining fixed-size and dynamic windows, adapting itself to the dynamics of the time series. Secondly, the prediction of convective cloud formation (which is a highly imbalance problem given that the number of convective cloud events is much lower than that of non-convective situations) is performed in two completely different ways: 1) tackling the problem as a multi-objective classification task by the use of multi-objective evolutionary artificial neural networks (MOEANNs), in which the two conflictive objectives are accuracy of the minority class and the global accuracy, and 2) tackling the problem from the OC point of view, in which, in order to reduce the imbalance degree, an oversampling approach is proposed along with the use of OC techniques. Thirdly, the prediction of solar radiation is carried out by means of evolutionary artificial neural networks (EANNs) with different combinations of basis functions in the hidden and output layers. Finally, the last challenging problem is the prediction of energy flux from waves and tides. For this, a multitask EANN has been proposed aiming to predict the energy flux at several prediction time horizons (from 6h to 48h). All these proposals and techniques have been corroborated and discussed according to physical and atmospheric models. The work developed in this Thesis is supported by 11 JCR-indexed papers in international journals (7 Q1, 3 Q2, 1 Q3), 11 papers in international conferences, and 4 papers in national conferences