Adapting Multicomponent Predictive Systems using Hybrid Adaptation Strategies with Auto-WEKA in Process Industry

Automation of composition and optimisation of multicomponent predictive systems (MCPSs) made of a number of preprocessing steps and predictive models is a challenging problem that has been addressed in recent works. However, one of the current challenges is how to adapt these systems in dynamic environments where data is changing over time. In this work we propose a hybrid approach combining different adaptation strategies with the Bayesian optimisation techniques for parametric, structural and hyperparameter optimisation of entire MCPSs. Experiments comparing different adaptation strategies have been performed on 7 datasets from real chemical production processes. Experimental analysis shows that optimisation of entire MCPSs as a method of adaptation to changing environments is feasible and that hybrid strategies perform better in most of the analysed cases

Automatic and adaptive preprocessing for the development of predictive models.

In recent years, there has been an increasing interest in extracting valuable information from large amounts of data. This information can be useful for making predictions about the future or inferring unknown values. There exists a multitude of predictive models for the most common tasks of classification and regression. However, researchers often assume that data is clean and far too little attention has been paid to data pre-processing. Despite the fact that there are a number of methods for accomplishing individual pre-processing tasks (e.g. outlier detection or feature selection), the effort of performing comprehensive data preparation and cleaning can take between 60% and 80% of the whole data mining process time. One of the goals of this research is to speed up this process and make it more efficient. To this end, an approach for automating the selection and optimisation of multiple preprocessing methods and predictors has been proposed. The combination of multiple data mining methods forming a workflow is known as Multi-Component Predictive System (MCPS). There are multiple software platforms like Weka and RapidMiner to create and run MCPSs including a large variety of pre-processing methods and predictors. There is, however, no common mathematical representation of MCPSs. An objective of this thesis is to establish a common representation framework of MCPSs. This will allow validating workflows before beginning the implementation phase with any particular platform. The validation of workflows becomes even more relevant when considering the automatic generation of MCPSs. In order to automate the composition and optimisation of MCPSs, a search space is defined consisting of a number of preprocessing methods, predictive models and their hyperparameters. Then, the space is explored using a Bayesian optimisation strategy within a given time or computational budget. As a result, a parametrised sequence of methods is returned which after training form a complete predictive system. The whole process is data-driven and does not require human intervention once it has been started. The generated predictive system can then be used to make predictions in an online scenario. However, it is possible that the nature of the input data changes over time. As a result, predictive models may need to be updated to capture the new characteristics of the data in order to reduce the loss of predictive performance. Similarly, preprocessing methods may have to be adapted as well. A novel hybrid strategy combining Bayesian optimisation and common adaptive techniques is proposed to automatically adapt MCPSs. This approach performs a global adaptation of the MCPS. However, in some situations, it could be costly to update the whole predictive system when maybe just a little adjustment is needed. The consequences of adapting a single component can, however, be significant. This thesis also analyses the impact of adapting individual components in an MCPS and proposes an approach to propagate changes through the system. This thesis was initiated due to a joint research project with a chemical production company, which has provided several datasets with common raw data issues in the process industry. The final part of this thesis evaluates the feasibility of applying such automatic techniques for building and maintaining predictive models for real chemical production processes

Automatic Composition and Optimization of Multicomponent Predictive Systems With an Extended Auto-WEKA.

Composition and parameterization of multicomponent predictive systems (MCPSs) consisting of chains of data transformation steps are a challenging task. Auto-WEKA is a tool to automate the combined algorithm selection and hyperparameter (CASH) optimization problem. In this paper, we extend the CASH problem and Auto-WEKA to support the MCPS, including preprocessing steps for both classification and regression tasks. We define the optimization problem in which the search space consists of suitably parameterized Petri nets forming the sought MCPS solutions. In the experimental analysis, we focus on examining the impact of considerably extending the search space (from approximately 22,000 to 812 billion possible combinations of methods and categorical hyperparameters). In a range of extensive experiments, three different optimization strategies are used to automatically compose MCPSs for 21 publicly available data sets. The diversity of the composed MCPSs found is an indication that fully and automatically exploiting different combinations of data cleaning and preprocessing techniques is possible and highly beneficial for different predictive models. We also present the results on seven data sets from real chemical production processes. Our findings can have a major impact on the development of high-quality predictive models as well as their maintenance and scalability aspects needed in modern applications and deployment scenarios

Automated Adaptation Strategies for Stream Learning

Automation of machine learning model development is increasingly becoming an established research area. While automated model selection and automated data pre-processing have been studied in depth, there is, however, a gap concerning automated model adaptation strategies when multiple strategies are available. Manually developing an adaptation strategy can be time consuming and costly. In this paper we address this issue by proposing the use of flexible adaptive mechanism deployment for automated development of adaptation strategies. Experimental results after using the proposed strategies with five adaptive algorithms on 36 datasets confirm their viability. These strategies achieve better or comparable performance to the custom adaptation strategies and the repeated deployment of any single adaptive mechanism

AutoWeka4MCPS-AVATAR: Accelerating Automated Machine Learning Pipeline Composition and Optimisation

Automated machine learning pipeline (ML) composition and optimisation aim at automating the process of finding the most promising ML pipelines within allocated resources (i.e., time, CPU and memory). Existing methods, such as Bayesian-based and genetic-based optimisation, which are implemented in Auto-Weka, Auto-sklearn and TPOT, evaluate pipelines by executing them. Therefore, the pipeline composition and optimisation of these methods frequently require a tremendous amount of time that prevents them from exploring complex pipelines to find better predictive models. To further explore this research challenge, we have conducted experiments showing that many of the generated pipelines are invalid in the first place, and attempting to execute them is a waste of time and resources. To address this issue, we propose a novel method to evaluate the validity of ML pipelines, without their execution, using a surrogate model (AVATAR). The AVATAR generates a knowledge base by automatically learning the capabilities and effects of ML algorithms on datasets' characteristics. This knowledge base is used for a simplified mapping from an original ML pipeline to a surrogate model which is a Petri net based pipeline. Instead of executing the original ML pipeline to evaluate its validity, the AVATAR evaluates its surrogate model constructed by capabilities and effects of the ML pipeline components and input/output simplified mappings. Evaluating this surrogate model is less resource-intensive than the execution of the original pipeline. As a result, the AVATAR enables the pipeline composition and optimisation methods to evaluate more pipelines by quickly rejecting invalid pipelines. We integrate the AVATAR into the sequential model-based algorithm configuration (SMAC). Our experiments show that when SMAC employs AVATAR, it finds better solutions than on its own.Comment: arXiv admin note: substantial text overlap with arXiv:2001.1115

The Technological Emergence of AutoML: A Survey of Performant Software and Applications in the Context of Industry

With most technical fields, there exists a delay between fundamental academic research and practical industrial uptake. Whilst some sciences have robust and well-established processes for commercialisation, such as the pharmaceutical practice of regimented drug trials, other fields face transitory periods in which fundamental academic advancements diffuse gradually into the space of commerce and industry. For the still relatively young field of Automated/Autonomous Machine Learning (AutoML/AutonoML), that transitory period is under way, spurred on by a burgeoning interest from broader society. Yet, to date, little research has been undertaken to assess the current state of this dissemination and its uptake. Thus, this review makes two primary contributions to knowledge around this topic. Firstly, it provides the most up-to-date and comprehensive survey of existing AutoML tools, both open-source and commercial. Secondly, it motivates and outlines a framework for assessing whether an AutoML solution designed for real-world application is 'performant'; this framework extends beyond the limitations of typical academic criteria, considering a variety of stakeholder needs and the human-computer interactions required to service them. Thus, additionally supported by an extensive assessment and comparison of academic and commercial case-studies, this review evaluates mainstream engagement with AutoML in the early 2020s, identifying obstacles and opportunities for accelerating future uptake

Multiple adaptive mechanisms for predictive models on streaming data.

Making predictions on non-stationary streaming data remains a challenge in many application areas. Changes in data may cause a decrease in predictive accuracy, which in a streaming setting require a prompt response. In recent years many adaptive predictive models have been proposed for dealing with these issues. Most of these methods use more than one adaptive mechanism, deploying all of them at the same time at regular intervals or in some other fixed manner. However, this manner is often determined in an ad-hoc way, as the effects of adaptive mechanisms are largely unexplored. This thesis therefore investigates different aspects of adaptation with multiple adaptive mechanisms with the aim to increase knowledge in the area, and propose heuristic approaches for more accurate adaptive predictive models. This is done by systematising and formalising the “adaptive mechanism” notion, proposing a categorisation of adaptive mechanisms and a metric to measure their usefulness, comparing the results after deployment of different orders of adaptive mechanisms during the run of the predictive method, and suggesting techniques on how to select the most appropriate adaptive mechanisms. The literature review suggests that during the prediction process, adaptive mechanisms are selected to be deployed in a certain order which is usually fixed beforehand at the design time of the algorithm. For this reason, it was investigated whether changing the selection method for the adaptive mechanisms significantly affects predictive accuracy and whether there are certain deployment orders which provide better results than others. Commonly used adaptive mechanism selection methods are then examined and new methods are proposed. A novel regression ensemble method which uses several common adaptive mechanisms has been developed to be used as a vehicle for the experimentation. The predictive accuracy and behaviour of adaptive mechanisms while predicting on different real world datasets from the process industry were analysed. Empirical results suggest that different selection of adaptive mechanisms result in significantly different performance. It has been found that while some adaptive mechanisms adapt the predictive model better than others, there is none which is the best at all times. Finally, flexible orders of adaptive mechanisms generated using the proposed selection techniques often result in significantly more accurate models than fixed orders commonly used in literature

Personality-based recommendation: human curiosity applied to recommendation systems using implicit information from social networks

Tesis por compendioEn el día a día, las personas suelen confiar en recomendaciones, tradicionalmente aportadas por otras personas (familia, amigos, etc.) para sus decisiones más variadas. En el mundo digital esto no es diferente, dado que los sistemas de recomendación están presentes en todas partes y de modo transparente. El principal objetivo de estos sistemas es el de ayudar en el proceso de toma de decisiones, generando recomendaciones de su interés y basadas en sus gustos. Dichas recomendaciones van desde productos en sitios web de comercio electrónico, como libros o lugares a visitar, además de qué comer o cuánto tiempo uno debe caminar al día para tener una vida sana, con quién salir o a quién seguir en las redes sociales. Esta es un área en ascensión. Por un lado, tenemos cada vez más usuarios en internet cuya vida está digitalizada, dado que lo que se hace en el "mundo real" está representado en cierto modo en el "mundo digital". Por otro lado, sufrimos una sobrecarga de información, que puede mitigarse mediante el uso de un sistema de recomendación. Sin embargo, estos sistemas también enfrentan algunos problemas, como el problema del arranque en frío y su necesidad de ser cada vez más "humanos", "personalizados" y "precisos" para satisfacer las exigencias de usuarios y empresas. En este desafiante escenario, los sistemas de recomendación basados en la personalidad se están estudiando cada vez más, ya que son capaces de enfrentar esos problemas. Algunos proyectos recientes proponen el uso de la personalidad humana en los recomendadores, ya sea en su conjunto o individualmente por rasgos. Esta tesis está dedicada a este nuevo área de recomendación basada en la personalidad, centrándose en uno de sus rasgos más importantes, la curiosidad. Además, para explotar la información ya existente en internet, obtendremos de forma implícita información de las redes sociales. Por lo tanto, este trabajo tiene como objetivo proporcionar una mejor experiencia al usuario final a través de un nuevo enfoque que ofrece una alternativa a algunos de los retos identificados en los sistemas de recomendación basados en la personalidad. Entre estas mejoras, el uso de las redes sociales para alimentar los sistemas de recomendación reduce el problema del arranque en frío y, al mismo tiempo, proporciona datos valiosos para la predicción de la personalidad humana. Por otro lado, la curiosidad no ha sido utilizada por ninguno de los sistemas de recomendación estudiados; casi todos han usado la personalidad general de un individuo a través de los Cinco Grandes rasgos de la personalidad. Sin embargo, los estudios psicológicos confirman que la curiosidad es un rasgo relevante en el proceso de elegir un item, cuestión directamente relacionada con los sistemas de recomendación. En resumen, creemos que un sistema de recomendación que mida implícitamente la curiosidad y la utilice en el proceso de recomendar nuevos ítems, especialmente en el sector turístico, podría claramente mejorar la capacidad de estos sistemas en términos de precisión, serendipidad y novedad, permitiendo a los usuarios obtener niveles positivos de satisfacción con las recomendaciones. Esta tesis realiza un estudio exhaustivo del estado del arte, donde destacamos trabajos sobre sistemas de recomendación, la personalidad humana desde el punto de vista de la psicología tradicional y positiva y finalmente cómo se combinan ambos aspectos. Luego, desarrollamos una aplicación en línea capaz de extraer implícitamente información del perfil de usuario en una red social, generando predicciones de uno o más rasgos de su personalidad. Finalmente, desarrollamos el sistema CURUMIM, capaz de generar recomendaciones en línea con diferentes propiedades, combinando la curiosidad y algunas características sociodemográficas (como el nivel de educación) extraídas de Facebook. El sistema ha sido probado y evaluado en el contexto turístico por usuarios rEn el dia a dia, les persones solen confiar en recomanacions, tradicionalment aportades per altres persones (família, amics, etc.) per a les seues decisions més variades. En el món digital això no és diferent, atès que els sistemes de recomanació estan presents a tot arreu i de manera transparent. El principal objectiu d'aquests sistemes és el d'ajudar en el procés de presa de decisions, generant recomanacions del seu interès i basades en els seus gustos. Aquestes recomanacions van des de productes en pàgines web de comerç electrònic, com a llibres o llocs a visitar, a més de què menjar o quant temps una persona ha de caminar al dia per a tindre una vida sana, amb qui eixir o a qui seguir en les xarxes socials. Aquesta és una àrea en ascensió. D'una banda, tenim cada vegada més usuaris en internet la vida de les quals està digitalitzada, atès que el que es fa en el "món real" està representat en certa manera en el "món digital". D'altra banda, patim una sobrecàrrega d'informació, que pot mitigar-se mitjançant l'ús d'un sistema de recomanació. No obstant això, aquests sistemes també enfronten alguns problemes, com el problema de l'arrencada en fred i la seua necessitat de ser cada vegada més "humans", "personalitzats" i "precisos" per a satisfer les exigències d'usuaris i empreses. En aquest desafiador escenari, els sistemes de recomanació basats en la personalitat s'estan estudiant cada vegada més, ja que són capaços d'enfrontar eixos problemes. Alguns projectes recents proposen l'ús de la personalitat humana en els recomendadors, ja siga en el seu conjunt o individualment per trets. Aquesta tesi està dedicada a aquest nou àrea de recomanació basada en la personalitat, centrant-se en un dels seus trets més importants, la curiositat. A més, per a explotar la informació ja existent en internet, obtindrem de forma implícita informació de les xarxes socials. Per tant, aquest treball té com a objectiu proporcionar una millor experiència a l'usuari final a través d'un nou enfocament que ofereix una alternativa a alguns dels reptes identificats en els sistemes de recomanació basats en la personalitat. Entre aquestes millores, l'ús de les xarxes socials per a alimentar els sistemes de recomanació redueix el problema de l'arrencada en fred i, al mateix temps, proporciona dades valuoses per a la predicció de la personalitat humana. D'altra banda, la curiositat no ha sigut utilitzada per cap dels sistemes de recomanació estudiats; quasi tots han usat la personalitat general d'un individu a través dels Cinc Grans trets de la personalitat. No obstant això, els estudis psicològics confirmen que la curiositat és un tret rellevant en el procés de triar un item, qüestió directament relacionada amb els sistemes de recomanació. En resum, creiem que un sistema de recomanació que mesure implícitament la curiositat i la utilitze en el procés de recomanar nous ítems, especialment en el sector turístic, podria clarament millorar la capacitat d'aquests sistemes en termes de precisió, sorpresa i novetat, permetent als usuaris obtindre nivells positius de satisfacció amb les recomanacions. Aquesta tesi realitza un estudi exhaustiu de l'estat de l'art, on destaquem treballs sobre sistemes de recomanació, la personalitat humana des del punt de vista de la psicologia tradicional i positiva i finalment com es combinen tots dos aspectes. Després, desenvolupem una aplicació en línia capaç d'extraure implícitament informació del perfil d'usuari en una xarxa social, generant prediccions d'un o més trets de la seua personalitat. Finalment, desenvolupem el sistema CURUMIM, capaç de generar recomanacions en línia amb diferents propietats, combinant la curiositat i algunes característiques sociodemogràfiques (com el nivell d'educació) extretes de Facebook. El sistema ha sigut provat i avaluat en el context turístic per usuaris reals. Els resultats demostren la seua capacitat perIn daily life, people usually rely on recommendations, traditionally given by other people (family, friends, etc.) for their most varied decisions. In the digital world, this is not different, given that recommender systems are present everywhere in such a way that we no longer realize. The main goal of these systems is to assist users in the decision-making process, generating recommendations that are of their interest and based on their tastes. These recommendations range from products in e-commerce websites, like books to read or places to visit to what to eat or how long one should walk a day to have a healthy life, who to date or who one should follow on social networks. And this is an increasing area. On the one hand, we have more and more users on the internet whose life is somewhat digitized, given than what one does in the "real world" is represented in a certain way in the "digital world". On the other hand, we suffer from information overload, which can be mitigated by the use of recommendation systems. However, these systems also face some problems, such as the cold start problem and their need to be more and more "human", "personalised" and "precise" in order to meet the yearning of users and companies. In this challenging scenario, personality-based recommender systems are being increasingly studied, since they are able to face these problems. Some recent projects have proposed the use of the human personality in recommenders, whether as a whole or individually by facet in order to meet those demands. Therefore, this thesis is devoted to this new area of personality-based recommendation, focusing on one of its most important traits, the curiosity. Additionally, in order to exploit the information already present on the internet, we will implicitly obtain information from social networks. Thus, this work aims to build a better experience for the end user through a new approach that offers an option for some of the gaps identified in personality-based recommendation systems. Among these gap improvements, the use of social networks to feed the recommender systems soften the cold start problem and, at the same time, it provides valuable data for the prediction of the human personality. Another found gap is that the curiosity was not used by any of the studied recommender systems; almost all of them have used the overall personality of an individual through the Big Five personality traits. However, psychological studies confirm that the curiosity is a relevant trait in the process of choosing an item, which is directly related to recommendation systems. In summary, we believe that a recommendation system that implicitly measures the curiosity and uses it in the process of recommending new items, especially in the tourism sector, could clearly improve the capacity of these systems in terms of accuracy, serendipity and novelty, allowing users to obtain positive levels of satisfaction with the recommendations. This thesis begins with an exhaustive study of the state of the art, where we highlight works about recommender systems, the human personality from the point of view of traditional and positive psychology and how these aspects are combined. Then, we develop an online application capable of implicitly extracting information from the user profile in a social network, thus generating predictions of one or more personality traits. Finally, we develop the CURUMIM system, able to generate online recommendations with different properties, combining the curiosity and some sociodemographic characteristics (such as level of education) extracted from Facebook. The system is tested and assessed within the tourism context by real users. The results demonstrate its ability to generate novel and serendipitous recommendations, while maintaining a good level of accuracy, independently of the degree of curiosity of the users.Menk Dos Santos, A. (2018). Personality-based recommendation: human curiosity applied to recommendation systems using implicit information from social networks [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/114798TESISCompendi

Advances in Artificial Intelligence: Models, Optimization, and Machine Learning

The present book contains all the articles accepted and published in the Special Issue “Advances in Artificial Intelligence: Models, Optimization, and Machine Learning” of the MDPI Mathematics journal, which covers a wide range of topics connected to the theory and applications of artificial intelligence and its subfields. These topics include, among others, deep learning and classic machine learning algorithms, neural modelling, architectures and learning algorithms, biologically inspired optimization algorithms, algorithms for autonomous driving, probabilistic models and Bayesian reasoning, intelligent agents and multiagent systems. We hope that the scientific results presented in this book will serve as valuable sources of documentation and inspiration for anyone willing to pursue research in artificial intelligence, machine learning and their widespread applications