Hybridization of machine learning for advanced manufacturing

Tesis por compendio de publicacioines[ES] En el contexto de la industria, hoy por hoy, los términos “Fabricación Avanzada”, “Industria 4.0” y “Fábrica Inteligente” están convirtiéndose en una realidad. Las empresas industriales buscan ser más competitivas, ya sea en costes, tiempo, consumo de materias primas, energía, etc. Se busca ser eficiente en todos los ámbitos y además ser sostenible. El futuro de muchas compañías depende de su grado de adaptación a los cambios y su capacidad de innovación. Los consumidores son cada vez más exigentes, buscando productos personalizados y específicos con alta calidad, a un bajo coste y no contaminantes. Por todo ello, las empresas industriales implantan innovaciones tecnológicas para conseguirlo. Entre estas innovaciones tecnológicas están la ya mencionada Fabricación Avanzada (Advanced Manufacturing) y el Machine Learning (ML). En estos campos se enmarca el presente trabajo de investigación, en el que se han concebido y aplicado soluciones inteligentes híbridas que combinan diversas técnicas de ML para resolver problemas en el campo de la industria manufacturera. Se han aplicado técnicas inteligentes tales como Redes Neuronales Artificiales (RNA), algoritmos genéticos multiobjetivo, métodos proyeccionistas para la reducción de la dimensionalidad, técnicas de agrupamiento o clustering, etc. También se han utilizado técnicas de Identificación de Sistemas con el propósito de obtener el modelo matemático que representa mejor el sistema real bajo estudio. Se han hibridado diversas técnicas con el propósito de construir soluciones más robustas y fiables. Combinando técnicas de ML específicas se crean sistemas más complejos y con una mayor capacidad de representación/solución. Estos sistemas utilizan datos y el conocimiento sobre estos para resolver problemas. Las soluciones propuestas buscan solucionar problemas complejos del mundo real y de un amplio espectro, manejando aspectos como la incertidumbre, la falta de precisión, la alta dimensionalidad, etc. La presente tesis cubre varios casos de estudio reales, en los que se han aplicado diversas técnicas de ML a distintas problemáticas del campo de la industria manufacturera. Los casos de estudio reales de la industria en los que se ha trabajado, con cuatro conjuntos de datos diferentes, se corresponden con: • Proceso de fresado dental de alta precisión, de la empresa Estudio Previo SL. • Análisis de datos para el mantenimiento predictivo de una empresa del sector de la automoción, como es la multinacional Grupo Antolin. Adicionalmente se ha colaborado con el grupo de investigación GICAP de la Universidad de Burgos y con el centro tecnológico ITCL en los casos de estudio que forman parte de esta tesis y otros relacionados. Las diferentes hibridaciones de técnicas de ML desarrolladas han sido aplicadas y validadas con conjuntos de datos reales y originales, en colaboración con empresas industriales o centros de fresado, permitiendo resolver problemas actuales y complejos. De esta manera, el trabajo realizado no ha tenido sólo un enfoque teórico, sino que se ha aplicado de modo práctico permitiendo que las empresas industriales puedan mejorar sus procesos, ahorrar en costes y tiempo, contaminar menos, etc. Los satisfactorios resultados obtenidos apuntan hacia la utilidad y aportación que las técnicas de ML pueden realizar en el campo de la Fabricación Avanzada

Advances in Robotics, Automation and Control

The book presents an excellent overview of the recent developments in the different areas of Robotics, Automation and Control. Through its 24 chapters, this book presents topics related to control and robot design; it also introduces new mathematical tools and techniques devoted to improve the system modeling and control. An important point is the use of rational agents and heuristic techniques to cope with the computational complexity required for controlling complex systems. Through this book, we also find navigation and vision algorithms, automatic handwritten comprehension and speech recognition systems that will be included in the next generation of productive systems developed by man

Machine learning approaches for lung cancer diagnosis.

The enormity of changes and development in the field of medical imaging technology is hard to fathom, as it does not just represent the technique and process of constructing visual representations of the body from inside for medical analysis and to reveal the internal structure of different organs under the skin, but also it provides a noninvasive way for diagnosis of various disease and suggest an efficient ways to treat them. While data surrounding all of our lives are stored and collected to be ready for analysis by data scientists, medical images are considered a rich source that could provide us with a huge amount of data, that could not be read easily by physicians and radiologists, with valuable information that could be used in smart ways to discover new knowledge from these vast quantities of data. Therefore, the design of computer-aided diagnostic (CAD) system, that can be approved for use in clinical practice that aid radiologists in diagnosis and detecting potential abnormalities, is of a great importance. This dissertation deals with the development of a CAD system for lung cancer diagnosis, which is the second most common cancer in men after prostate cancer and in women after breast cancer. Moreover, lung cancer is considered the leading cause of cancer death among both genders in USA. Recently, the number of lung cancer patients has increased dramatically worldwide and its early detection doubles a patient’s chance of survival. Histological examination through biopsies is considered the gold standard for final diagnosis of pulmonary nodules. Even though resection of pulmonary nodules is the ideal and most reliable way for diagnosis, there is still a lot of different methods often used just to eliminate the risks associated with the surgical procedure. Lung nodules are approximately spherical regions of primarily high density tissue that are visible in computed tomography (CT) images of the lung. A pulmonary nodule is the first indication to start diagnosing lung cancer. Lung nodules can be benign (normal subjects) or malignant (cancerous subjects). Large (generally defined as greater than 2 cm in diameter) malignant nodules can be easily detected with traditional CT scanning techniques. However, the diagnostic options for small indeterminate nodules are limited due to problems associated with accessing small tumors. Therefore, additional diagnostic and imaging techniques which depends on the nodules’ shape and appearance are needed. The ultimate goal of this dissertation is to develop a fast noninvasive diagnostic system that can enhance the accuracy measures of early lung cancer diagnosis based on the well-known hypotheses that malignant nodules have different shape and appearance than benign nodules, because of the high growth rate of the malignant nodules. The proposed methodologies introduces new shape and appearance features which can distinguish between benign and malignant nodules. To achieve this goal a CAD system is implemented and validated using different datasets. This CAD system uses two different types of features integrated together to be able to give a full description to the pulmonary nodule. These two types are appearance features and shape features. For the appearance features different texture appearance descriptors are developed, namely the 3D histogram of oriented gradient, 3D spherical sector isosurface histogram of oriented gradient, 3D adjusted local binary pattern, 3D resolved ambiguity local binary pattern, multi-view analytical local binary pattern, and Markov Gibbs random field. Each one of these descriptors gives a good description for the nodule texture and the level of its signal homogeneity which is a distinguishable feature between benign and malignant nodules. For the shape features multi-view peripheral sum curvature scale space, spherical harmonics expansions, and different group of fundamental geometric features are utilized to describe the nodule shape complexity. Finally, the fusion of different combinations of these features, which is based on two stages is introduced. The first stage generates a primary estimation for every descriptor. Followed by the second stage that consists of an autoencoder with a single layer augmented with a softmax classifier to provide us with the ultimate classification of the nodule. These different combinations of descriptors are combined into different frameworks that are evaluated using different datasets. The first dataset is the Lung Image Database Consortium which is a benchmark publicly available dataset for lung nodule detection and diagnosis. The second dataset is our local acquired computed tomography imaging data that has been collected from the University of Louisville hospital and the research protocol was approved by the Institutional Review Board at the University of Louisville (IRB number 10.0642). These frameworks accuracy was about 94%, which make the proposed frameworks demonstrate promise to be valuable tool for the detection of lung cancer

IMAGE UNDERSTANDING OF MOLAR PREGNANCY BASED ON ANOMALIES DETECTION

Cancer occurs when normal cells grow and multiply without normal control. As the cells multiply, they form an area of abnormal cells, known as a tumour. Many tumours exhibit abnormal chromosomal segregation at cell division. These anomalies play an important role in detecting molar pregnancy cancer. Molar pregnancy, also known as hydatidiform mole, can be categorised into partial (PHM) and complete (CHM) mole, persistent gestational trophoblastic and choriocarcinoma. Hydatidiform moles are most commonly found in women under the age of 17 or over the age of 35. Hydatidiform moles can be detected by morphological and histopathological examination. Even experienced pathologists cannot easily classify between complete and partial hydatidiform moles. However, the distinction between complete and partial hydatidiform moles is important in order to recommend the appropriate treatment method. Therefore, research into molar pregnancy image analysis and understanding is critical. The hypothesis of this research project is that an anomaly detection approach to analyse molar pregnancy images can improve image analysis and classification of normal PHM and CHM villi. The primary aim of this research project is to develop a novel method, based on anomaly detection, to identify and classify anomalous villi in molar pregnancy stained images. The novel method is developed to simulate expert pathologists’ approach in diagnosis of anomalous villi. The knowledge and heuristics elicited from two expert pathologists are combined with the morphological domain knowledge of molar pregnancy, to develop a heuristic multi-neural network architecture designed to classify the villi into their appropriated anomalous types. This study confirmed that a single feature cannot give enough discriminative power for villi classification. Whereas expert pathologists consider the size and shape before textural features, this thesis demonstrated that the textural feature has a higher discriminative power than size and shape. The first heuristic-based multi-neural network, which was based on 15 elicited features, achieved an improved average accuracy of 81.2%, compared to the traditional multi-layer perceptron (80.5%); however, the recall of CHM villi class was still low (64.3%). Two further textural features, which were elicited and added to the second heuristic-based multi-neural network, have improved the average accuracy from 81.2% to 86.1% and the recall of CHM villi class from 64.3% to 73.5%. The precision of the multi-neural network II has also increased from 82.7% to 89.5% for normal villi class, from 81.3% to 84.7% for PHM villi class and from 80.8% to 86% for CHM villi class. To support pathologists to visualise the results of the segmentation, a software tool, Hydatidiform Mole Analysis Tool (HYMAT), was developed compiling the morphological and pathological data for each villus analysis

Enhancing the bees algorithm using the traplining metaphor

This work aims to improve the performance of the Bees Algorithm (BA), particularly in terms of simplicity, accuracy, and convergence. Three improvements were made in this study as a result of bees’ traplining behaviour. The first improvement was the parameter reduction of the Bees Algorithm. This strategy recruits and assigns worker bees to exploit and explore all patches. Both searching processes are assigned using the Triangular Distribution Random Number Generator. The most promising patches have more workers and are subject to more exploitation than the less productive patches. This technique reduced the original parameters into two parameters. The results show that the Bi-BA is just as efficient as the basic BA, although it has fewer parameters. Following that, another improvement was proposed to increase the diversification performance of the Combinatorial Bees Algorithm (CBA). The technique employs a novel constructive heuristic that considers the distance and the turning angle of the bees’ flight. When foraging for honey, bees generally avoid making a sharp turn. By including this turning angle as the second consideration, it can control CBA’s initial solution diversity. Third, the CBA is strengthened to enable an intensification strategy that avoids falling into a local optima trap. The approach is based on the behaviour of bees when confronted with threats. They will keep away from re-visiting those flowers during the next bout for reasons like predators, rivals, or honey run out. The approach will remove temporarily threatened flowers from the whole tour, eliminating the sharp turn, and reintroduces them again to the habitual tour’s nearest edge. The technique could effectively achieve an equilibrium between exploration and exploitation mechanisms. The results show that the strategy is very competitive compared to other population-based nature-inspired algorithms. Finally, the enhanced Bees Algorithms are demonstrated on two real-world engineering problems, namely, Printed Circuit Board insertion sequencing and vehicles routing problem

Uncertainty modeling : fundamental concepts and models

This book series represents a commendable effort in compiling the latest developments on three important Engineering subjects: discrete modeling, inverse methods, and uncertainty structural integrity. Although academic publications on these subjects are plenty, this book series may be the first time that these modern topics are compiled together, grouped in volumes, and made available for the community. The application of numerical or analytical techniques to model complex Engineering problems, fed by experimental data, usually translated in the form of stochastic information collected from the problem in hand, is much closer to real-world situations than the conventional solution of PDEs. Moreover, inverse problems are becoming almost as common as direct problems, given the need in the industry to maintain current processes working efficiently, as well as to create new solutions based on the immense amount of information available digitally these days. On top of all this, deterministic analysis is slowly giving space to statistically driven structural analysis, delivering upper and lower bound solutions which help immensely the analyst in the decisionmaking process. All these trends have been topics of investigation for decades, and in recent years the application of these methods in the industry proves that they have achieved the necessary maturity to be definitely incorporated into the roster of modern Engineering tools. The present book series fulfills its role by collecting and organizing these topics, found otherwise scattered in the literature and not always accessible to industry. Moreover, many of the chapters compiled in these books present ongoing research topics conducted by capable fellows from academia and research institutes. They contain novel contributions to several investigation fields and constitute therefore a useful source of bibliographical reference and results repository. The Latin American Journal of Solids and Structures (LAJSS) is honored in supporting the publication of this book series, for it contributes academically and carries technologically significant content in the field of structural mechanics

Novel machine learning methods based on information theory

[Resumo] A aprendizaxe automática é a área da intelixencia artificial e da ciencia da computación que estuda algoritmos que aprenden a partir de datos, fan prediccións e producen comportamentos baseados en exemplos. Esta tesis desenvolve novos métodos de aprendiza.xe automática baseados en teoría da información [TI) e en information Iheoretic learning (ITL): (1) En primeiro lugar, utilizase TI para selección de características. Específicamente, se desenvolveo dous novos algoritmos. O primeiro ten en conta o coste (computacional. económico, etc.) de cada caracteristica -ademais da relevancia-. O segundo fai uso do concepto de enremble. moi comÚD en escenarios de clasificación, pero moi poueo explorado na literatura de selección de características. (2) En segundo lugar. se poden empregar conceptos de TI e ITL como unha función de erro alternativa. o cal permite a exploración doutro campo da literatura non moi estudado: a aproximación de modelado local. Específicamente, desenvólvese un novo algoritmo para clasificación. Este algoritmo está baseado na combinación de redes de neuronas por medio de modelado local e técnicas baseadas en ITL.[Resumen] El aprendizaje automático es el área de la inteligencia artificial y la ciencia de la computación que estudia los algoritmos que aprenden a partir de datos, realizan predicciones y producen comportamientos basados en ejemplos. Esta tesis desarrolla nuevos métodos de aprendizaje automático basados en teoría de la información (TI) y en information theoretic learning (ITL): (1) En primer lugar, se utiliza TI para selección de características. Específicamente, se desarrollan dos nuevos algoritmos. El primero tiene en cuenta el coste (computacional, económico, etc.) de cada característica -además de la relevancia-. El segundo hace uso del concepto de ensemble, muy común en escenarios de clasificación, pero muy poco explorado en la literatura de selección de características. (2) En segundo lugar. se pueden emplear conceptos de TI e ITL como una función de error alternativa, lo cual permite la exploración de otro campo de la literatura no muy estudiado: la aproximación de modelado local. Especificamente, se desarrolla un nuevo algoritmo para clasificación. Este algoritmo está basado en la combinación de redes de neuronas por medio de modelado local y técnicas basadas en ITL.[Abstract] Machine learning is the area of artificial intelligence and computer science that studies algorithms that can learn from data, make predictions, and produce behaviors based on examples. This thesis develops new methods of machine learning based on infonnation theory (IT) and information tbeoretic leaming (ITL): (1) On the one band, IT is used for feature selection. Specifically, two new algorithms are developed. The first one takes into account the cost (computational, economic, etc.) of each feature -besides its relevance-. Tbe second one makes use of the concept of ensemble, quite common for c1assification scenarios, but very little explored in the literature of feature selection. (2) 0n the other band, IT and ITL concepts can be employed as an altemative crror function, thus allowing the exploration of another not very well studied field in the literature: {he local modeling approach. Specifically, a new algorithm for classification is developed. This algorithm is based on the combination of neural networks by means of local modeling and techniques based on ITL

A model driven approach to imbalanced data learning

Ph.DDOCTOR OF PHILOSOPH

Automatic inference of causal reasoning chains from student essays

While there has been an increasing focus on higher-level thinking skills arising from the Common Core Standards, many high-school and middle-school students struggle to combine and integrate information from multiple sources when writing essays. Writing is an important learning skill, and there is increasing evidence that writing about a topic develops a deeper understanding in the student. However, grading essays is time consuming for teachers, resulting in an increasing focus on shallower forms of assessment that are easier to automate, such as multiple-choice tests. Existing essay grading software has attempted to ease this burden but relies on shallow lexico-syntactic features and is unable to understand the structure or validity of a student’s arguments or explanations. Without the ability to understand a student’s reasoning processes, it is impossible to write automated formative assessment systems to assist students with improving their thinking skills through essay writing. In order to understand the arguments put forth in an explanatory essay in the science domain, we need a method of representing the causal structure of a piece of explanatory text. Psychologists use a representation called a causal model to represent a student\u27s understanding of an explanatory text. This consists of a number of core concepts, and a set of causal relations linking them into one or more causal chains, forming a causal model. In this thesis I present a novel system for automatically constructing causal models from student scientific essays using Natural Language Processing (NLP) techniques. The problem was decomposed into 4 sub-problems - assigning essay concepts to words, detecting causal-relations between these concepts, resolving coreferences within each essay, and using the structure of the whole essay to reconstruct a causal model. Solutions to each of these sub-problems build upon the predictions from the solutions to earlier problems, forming a sequential pipeline of models. Designing a system in this way allows later models to correct for false positive predictions from downstream models. However, this also has the disadvantage that errors made in earlier models can propagate through the system, negatively impacting the upstream models, and limiting their accuracy. Producing robust solutions for the initial 2 sub problems, detecting concepts, and parsing causal relations between them, was critical in building a robust system. A number of sequence labeling models were trained to classify the concepts associated with each word, with the most effective approach being a bidirectional recurrent neural network (RNN), a deep learning model commonly applied to word labeling problems. This is because the RNN used pre-trained word embeddings to better generalize to rarer words, and was able to use information from both ends of each sentence to infer a word\u27s concept. The concepts predicted by this model were then used to develop causal relation parsing models for detecting causal connections between these concepts. A shift-reduce dependency parsing model was trained using the SEARN algorithm and out-performed a number of other approaches by better utilizing the structure of the problem and directly optimizing the error metric used. Two pre-trained coreference resolution systems were used to resolve coreferences within the essays. However a word tagging model trained to predict anaphors combined with a heuristic for determining the antecedent out-performed these two systems. Finally, a model was developed for parsing a causal model from an entire essay, utilizing the solutions to the three previous problems. A beam search algorithm was used to produce multiple parses for each sentence, which in turn were combined to generate multiple candidate causal models for each student essay. A reranking algorithm was then used to select the optimal causal model from all of the generated candidates. An important contribution of this work is that it represents a system for parsing a complete causal model of a scientific essay from a student\u27s written answer. Existing systems have been developed to parse individual causal relations, but no existing system attempts to parse a sequence of linked causal relations forming a causal model from an explanatory scientific essay. It is hoped that this work can lead to the development of more robust essay grading software and formative assessment tools, and can be extended to build solutions for extracting causality from text in other domains. In addition, I also present 2 novel approaches for optimizing the micro-F1 score within the design of two of the algorithms studied: the dependency parser and the reranking algorithm. The dependency parser uses a custom cost function to estimate the impact of parsing mistakes on the overall micro-F1 score, while the reranking algorithm allows the micro-F1 score to be optimized by tuning the beam search parameter to balance recall and precision