Uncertainty and Interpretability Studies in Soft Computing with an Application to Complex Manufacturing Systems

In systems modelling and control theory, the benefits of applying neural networks have been extensively studied. Particularly in manufacturing processes, such as the prediction of mechanical properties of heat treated steels. However, modern industrial processes usually involve large amounts of data and a range of non-linear effects and interactions that might hinder their model interpretation. For example, in steel manufacturing the understanding of complex mechanisms that lead to the mechanical properties which are generated by the heat treatment process is vital. This knowledge is not available via numerical models, therefore an experienced metallurgist estimates the model parameters to obtain the required properties. This human knowledge and perception sometimes can be imprecise leading to a kind of cognitive uncertainty such as vagueness and ambiguity when making decisions. In system classification, this may be translated into a system deficiency - for example, small input changes in system attributes may result in a sudden and inappropriate change for class assignation. In order to address this issue, practitioners and researches have developed systems that are functional equivalent to fuzzy systems and neural networks. Such systems provide a morphology that mimics the human ability of reasoning via the qualitative aspects of fuzzy information rather by its quantitative analysis. Furthermore, these models are able to learn from data sets and to describe the associated interactions and non-linearities in the data. However, in a like-manner to neural networks, a neural fuzzy system may suffer from a lost of interpretability and transparency when making decisions. This is mainly due to the application of adaptive approaches for its parameter identification. Since the RBF-NN can be treated as a fuzzy inference engine, this thesis presents several methodologies that quantify different types of uncertainty and its influence on the model interpretability and transparency of the RBF-NN during its parameter identification. Particularly, three kind of uncertainty sources in relation to the RBF-NN are studied, namely: entropy, fuzziness and ambiguity. First, a methodology based on Granular Computing (GrC), neutrosophic sets and the RBF-NN is presented. The objective of this methodology is to quantify the hesitation produced during the granular compression at the low level of interpretability of the RBF-NN via the use of neutrosophic sets. This study also aims to enhance the disitnguishability and hence the transparency of the initial fuzzy partition. The effectiveness of the proposed methodology is tested against a real case study for the prediction of the properties of heat-treated steels. Secondly, a new Interval Type-2 Radial Basis Function Neural Network (IT2-RBF-NN) is introduced as a new modelling framework. The IT2-RBF-NN takes advantage of the functional equivalence between FLSs of type-1 and the RBF-NN so as to construct an Interval Type-2 Fuzzy Logic System (IT2-FLS) that is able to deal with linguistic uncertainty and perceptions in the RBF-NN rule base. This gave raise to different combinations when optimising the IT2-RBF-NN parameters. Finally, a twofold study for uncertainty assessment at the high-level of interpretability of the RBF-NN is provided. On the one hand, the first study proposes a new methodology to quantify the a) fuzziness and the b) ambiguity at each RU, and during the formation of the rule base via the use of neutrosophic sets theory. The aim of this methodology is to calculate the associated fuzziness of each rule and then the ambiguity related to each normalised consequence of the fuzzy rules that result from the overlapping and to the choice with one-to-many decisions respectively. On the other hand, a second study proposes a new methodology to quantify the entropy and the fuzziness that come out from the redundancy phenomenon during the parameter identification. To conclude this work, the experimental results obtained through the application of the proposed methodologies for modelling two well-known benchmark data sets and for the prediction of mechanical properties of heat-treated steels conducted to publication of three articles in two peer-reviewed journals and one international conference

Perpetual Learning Framework based on Type-2 Fuzzy Logic System for a Complex Manufacturing Process

This paper introduces a perpetual type-2 Neuro-Fuzzy modelling structure for continuous learning and its application to the complex thermo-mechanical metal process of steel Friction Stir Welding (FSW). The ‘perpetual’ property refers to the capability of the proposed system to continuously learn from new process data, in an incremental learning fashion. This is particularly important in industrial/manufacturing processes, as it eliminates the need to retrain the model in the presence of new data, or in the case of any process drift. The proposed structure evolves through incremental, hybrid (supervised/unsupervised) learning, and accommodates new sample data in a continuous fashion. The human-like information capture paradigm of granular computing is used along with an interval type-2 neural-fuzzy system to develop a modelling structure that is tolerant to the uncertainty in the manufacturing data (common challenge in industrial/manufacturing data). The proposed method relies on the creation of new fuzzy rules which are updated and optimised during the incremental learning process. An iterative pruning strategy in the model is then employed to remove any redundant rules, as a result of the incremental learning process. The rule growing/pruning strategy is used to guarantee that the proposed structure can be used in a perpetual learning mode. It is demonstrated that the proposed structure can effectively learn complex dynamics of input-output data in an adaptive way and maintain good predictive performance in the metal processing case study of steel FSW using real manufacturing dat

An entropy-based uncertainty measure for developing granular models

There are two main ways to construct Fuzzy Logic rule-based models: using expert knowledge and using data mining methods. One of the most important aspects of Granular Computing (GrC) is to discover and extract knowledge from raw data in the form of information granules. The knowledge gained from the GrC, the information granules, can be used in constructing the linguistic rule-bases of a Fuzzy-Logic based system. Algorithms for iterative data granulation in the literature, so far, do not account for data uncertainty during the granulation process. In this paper, the uncertainty during the data granulation process is captured using the fundamental concept in information theory, entropy. In the proposed GrC algorithm, data granules are defined as information objects, hence the entropy measure being used in this research work is to capture the uncertainty in the data vectors resulting from the merging of the information granules. The entropy-based uncertainty measure is used to guide the iterative granulation process, hence promoting the formation of new granules with less uncertainty. The enhanced information granules are then being translated into a Fuzzy Logic inference system. The effectiveness of the proposed approach is demonstrated using established datasets

A new divergence measure based on fuzzy TOPSIS for solving staff performance appraisal

Various divergence measure methods have been used in many applications of fuzzy set theory for calculating the discrimination between two objects. This paper aims to develop a novel divergence measure incorporated with the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) method, along with the discussions of its properties. Since ambiguity or uncertainty is an inevitable characteristic of multi-criteria decision-making (MCDM) problems, the fuzzy concept is utilised to convert linguistic expressions into triangular fuzzy numbers. A numerical example of a staff performance appraisal is given to demonstrate suggested method's effectiveness and practicality. Outcomes from this study were compared with various MCDM techniques in terms of correlation coefficients and central processing unit (CPU) time. From the results, there is a slight difference in the ranking order between the proposed method and the other MCDM methods as all the correlation coefficient values are more than 0.9. It is also discovered that CPU time of the proposed method is the lowest compared to the other divergence measure techniques. Hence, the proposed method provides a more sensible and feasible solutions than its counterparts

An evolving feature weighting framework for radial basis function neural network models

Via Granular Computing (GrC), one can create effective computational frameworks for obtaining information from data, motivated by the human perception of combining similar objects. Combining knowledge gained via GrC with a Fuzzy inference engine (Neural-Fuzzy) enable us to develop a transparent system. While weighting variables based on their importance during the iterative data granulation process has been proposed before (W-GrC), there is no work in the literature to demonstrate effectiveness and impact on Type-2 Fuzzy Logic systems (T2-FLS). The main contribution of this paper is to extend W-GrC, for the first time, to both Type-1 and Type-2 models known as Radial Basis Function Neural Network (RBFNN) and General Type-2 Radial Basis Function Neural Network (GT2-RBFNN). The proposed framework is validated using popular datasets: Iris, Wine, Breast Cancer, Heart and Cardiotocography. Results show that with the appropriate selection of feature weight parameter, the new computational framework achieves better classification accuracy outcomes. In addition, we also introduce in this research work an investigation on the modelling structure's interpretability (via Nauck's index) where it is shown that a good balance of interpretability and accuracy can be maintained

Aproximaciones bioinformáticas para identificación de perfiles epigenéticos en procesos neuropatológicos

Degenerative neurological diseases, such as Alzheimer, Multiple Sclerosis or Huntington Disease, are illnesses that are not well-known while at the same time having a significant impact on the quality of life of the patients and their survival. The focus of this dissertation is finding biomarkers for the identification of these diseases, ideally in a rapid a reliable manner. The analysis was carried out using DNA CpG methylation data. In recent years there has been very significant technological improvements. It is currently possible to obtain the methylation levels for hundreds of thousands of CpG in a patient in a fast and reliable manner. It is however challenging to analyze these amounts of new data. A reasonable approach to tackle this issue is using machine learning techniques that have proven useful in many other fields. In this dissertation I developed a nonlinear approach to identifying combinations of CpGs DNA methylation data, as biomarkers for Alzheimer (AD) disease. It will be shown that this approach increases the accuracy of the detection on patients with AD when compared to directly using all the data available. I also analyzed the case of Huntington Disease (HD).Using nonlinear techniques I was able to reduce the number of CpGs considered from hundreds of thousands to 237 using a non-linear approach. It will be shown that using only these 237 CpGs and non-linear techniques such as artificial neural networks makes it possible to accurately differentiate between control and HD patients. Additionally, in this dissertation I present a technique, based on the concept of Shannon Entropy, to select CpGs as inputs for non-linear classification algorithms. It will be shown that this approach generates accurate classifications that are a statistically significant improvement over using all the data available or randomly selecting the same number of CpGs. The results seems to clearly illustrate that the analysis of the DNA methylation data, for the identification of patients suffering from the degenerative neurological diseases above mentioned, needs to be carefully carry out. Having the possibility of analyzing hundreds of thousands of CpGs level does not necessarily translate into better results as some of these levels might be unrelated and only adding noise to the analysis. It will be shown that the proposed algorithms generate accurate results while at the same time decreasing the number of CpGs used. For instance, in the case of Alzheimer the results obtained with the proposed algorithm generate a sensitivity of 0.9007 and a specificity of 0.9485. One of the underlying expectations is that in the future there will be curative treatments for these illnesses, which do not currently exists. It is also assumed that early detection, similarly to many other diseases, might be important when such treatments appear. Using the current technology it is relatively simple to analyze DNA methylation data and hence it can become an interesting biomarker in the context of these illnesses.Las enfermedades neurológicas degenerativas, como el Alzheimer, la Esclerosis Múltiple o la Enfermedad de Huntington son enfermedades que aún no son del todo conocidas y, al mismo tiempo, tienen un gran impacto en la calidad de vida del paciente y en su supervivencia. El enfoque de esta tesis es encontrar biomarcadores para la identificación de estas enfermedades, idealmente de una manera rápida y precisa. El análisis se llevó a cabo utilizando datos de metilación de ADN CpG. En los últimos años se han producido mejoras tecnológicas muy significativas. Actualmente es posible obtener los niveles de metilación para cientos de miles de CpG en un paciente de una manera rápida y confiable. Sin embargo, es difícil analizar estas cantidades de nuevos datos. Un enfoque razonable para abordar este problema es el uso de técnicas de aprendizaje automático que han demostrado ser útiles en muchos otros campos. En esta tesis doctoral desarrolle un enfoque no lineal para identificar combinaciones de datos de metilación del ADN (CpGs), como biomarcadores para la enfermedad de Alzheimer (EA). Se demostrará que este algoritmo aumenta la precisión de la detección en pacientes con EA en comparación con el uso directo de todos los datos disponibles. También analice el caso de la enfermedad de Huntington (EH). Usando técnicas no lineales pude reducir el número de CpG considerados de cientos de miles a 237 utilizando también un enfoque no lineal. Se demostrará que el uso de solo estos 237 CpG y técnicas no lineales como las redes neuronales artificiales permite diferenciar con precisión entre pacientes de control y EH. Adicionalmente, en esta tesis presento una técnica, basada en el concepto de Entropía de Shannon, para seleccionar CpGs como entradas para algoritmos de clasificación no lineal. Se demostrará que este enfoque genera clasificaciones precisas con una mejora estadísticamente significativa sobre el uso de todos los datos disponibles o la selección aleatoria del mismo número de CpG. Los resultados parecen ilustrar claramente que el análisis de los datos de metilación del ADN, para la identificación de pacientes que sufren de la enfermedad neurológica degenerativa antes mencionada, debe llevarse a cabo cuidadosamente. Tener la posibilidad de analizar cientos de miles de niveles de CpG no necesariamente se traduce en mejores resultados, ya que algunos de estos niveles pueden no estar relacionados y solo agregar ruido al análisis. Se demostrará que los algoritmos propuestos generan resultados precisos y, al mismo tiempo, disminuyen el número de CpG utilizados. Por ejemplo, en el caso del Alzheimer los resultados obtenidos con el algoritmo propuesto generan una sensibilidad de 0,9007 y una especificidad de 0,9485. Una de las expectativas subyacentes es que en el futuro habrá tratamientos curativos para estas enfermedades, que actualmente no existen. También se supone que la detección temprana, de manera similar a muchas otras enfermedades, podría ser importante cuando aparecen tales tratamientos. Utilizando la tecnología actual, es relativamente simple analizar los datos de metilación del ADN y, por lo tanto, puede convertirse en un biomarcador interesante en el contexto de estas enfermedades

A computational approach to analyzing and detecting trans-exclusionary radical feminists (TERFs) on Twitter

Within the realm of abusive content detection for social media, little research has been conducted on the transphobic hate group known as trans-exclusionary radical feminists (TERFs). The community engages in harmful behaviors such as targeted harassment of transgender people on Twitter, and perpetuates transphobic rhetoric such as denial of trans existence under the guise of feminism. This thesis analyzes the network of the TERF community on Twitter, by discovering several sub-communities as well as modeling the topics of their tweets. We also introduce TERFSPOT, a classifier for predicting whether a Twitter user is a TERF or not, based on a combination of network and textual features. The contributions of this work are twofold: we conduct the first large-scale computational analysis of the TERF hate group on Twitter, and demonstrate a classifier with a 90% accuracy for identifying TERFs

Uncertainty Management of Intelligent Feature Selection in Wireless Sensor Networks

Wireless sensor networks (WSN) are envisioned to revolutionize the paradigm of monitoring complex real-world systems at a very high resolution. However, the deployment of a large number of unattended sensor nodes in hostile environments, frequent changes of environment dynamics, and severe resource constraints pose uncertainties and limit the potential use of WSN in complex real-world applications. Although uncertainty management in Artificial Intelligence (AI) is well developed and well investigated, its implications in wireless sensor environments are inadequately addressed. This dissertation addresses uncertainty management issues of spatio-temporal patterns generated from sensor data. It provides a framework for characterizing spatio-temporal pattern in WSN. Using rough set theory and temporal reasoning a novel formalism has been developed to characterize and quantify the uncertainties in predicting spatio-temporal patterns from sensor data. This research also uncovers the trade-off among the uncertainty measures, which can be used to develop a multi-objective optimization model for real-time decision making in sensor data aggregation and samplin