Generation of Explicit Knowledge from Empirical Data through Pruning of Trainable Neural Networks

This paper presents a generalized technology of extraction of explicit knowledge from data. The main ideas are 1) maximal reduction of network complexity (not only removal of neurons or synapses, but removal all the unnecessary elements and signals and reduction of the complexity of elements), 2) using of adjustable and flexible pruning process (the pruning sequence shouldn't be predetermined - the user should have a possibility to prune network on his own way in order to achieve a desired network structure for the purpose of extraction of rules of desired type and form), and 3) extraction of rules not in predetermined but any desired form. Some considerations and notes about network architecture and training process and applicability of currently developed pruning techniques and rule extraction algorithms are discussed. This technology, being developed by us for more than 10 years, allowed us to create dozens of knowledge-based expert systems. In this paper we present a generalized three-step technology of extraction of explicit knowledge from empirical data.Comment: 9 pages, The talk was given at the IJCNN '99 (Washington DC, July 1999

CBR and MBR techniques: review for an application in the emergencies domain

The purpose of this document is to provide an in-depth analysis of current reasoning engine practice and the integration strategies of Case Based Reasoning and Model Based Reasoning that will be used in the design and development of the RIMSAT system. RIMSAT (Remote Intelligent Management Support and Training) is a European Commission funded project designed to: a.. Provide an innovative, 'intelligent', knowledge based solution aimed at improving the quality of critical decisions b.. Enhance the competencies and responsiveness of individuals and organisations involved in highly complex, safety critical incidents - irrespective of their location. In other words, RIMSAT aims to design and implement a decision support system that using Case Base Reasoning as well as Model Base Reasoning technology is applied in the management of emergency situations. This document is part of a deliverable for RIMSAT project, and although it has been done in close contact with the requirements of the project, it provides an overview wide enough for providing a state of the art in integration strategies between CBR and MBR technologies.Postprint (published version

Ship target recognition

Includes bibliographical references.In this report the classification of ship targets using a low resolution radar system is investigated. The thesis can be divided into two major parts. The first part summarizes research into the applications of neural networks to the low resolution non-cooperative ship target recognition problem. Three very different neural architectures are investigated and compared, namely; the Feedforward Network with Back-propagation, Kohonen's Supervised Learning Vector Quantization Network, and Simpson's Fuzzy Min-Max neural network. In all cases, pre-processing in the form of the Fourier-Modified Discrete Mellin Transform is used as a means of extracting feature vectors which are insensitive to the aspect angle of the radar. Classification tests are based on both simulated and real data. Classification accuracies of up to 93 are reported. The second part is of a purely investigative nature, and summarizes a body of research aimed at exploring new ground. The crux of this work is centered on the proposal to use synthetic range profiling in order to achieve a much higher range resolution (and hence better classification accuracies). Included in this work is a comprehensive investigation into the use of super-resolution and noise reducing eigendecomposition techniques. Algorithms investigated include the Principal Eigenvector Method, the Total Least Squares Method, and the MUSIC method. A final proposal for future research and development concerns the use of time domain averaging to improve the classification performance of the radar system. The use of an iterative correlation algorithm is investigated

A Neuro-Fuzzy Approach to Screening Reservoir Candidates for EOR

The authors are grateful to Prof. Elmira Ramazanova of the Scientific Research Institute Baku, Azerbaijan, for facilitating the forum for the discussion of this project. The authors also thank the editor of Oil and Gas Journal for the permission to use the worldwide EOR data.Peer reviewedPublisher PD

DATA CLASSIFICATION SYSTEM WITH FUZZY NEURAL BASED APPROACH

Knowledge Discovery in Database and Data Mining use techniques derived from machine learning, visualization and statistics to investigate real world data. Their aim is to discover patterns within the data which are new, statistically valid, interesting and understandable. In recent years, there has been an explosion in computation and information technology. With it have come vast amounts of data. Lying hidden in all this data is potentially useful information that is rarely made explicit or taken advantage. New tools based both on clever applications of established algorithms and on new methodologies, empower us to do entirely new things. In this context, data mining has arisen as an important research area that helps to reveal the hidden interesting information from the rawdatacollected. The project demonstrates how data mining can address the need of business intelligence in the process of decision making. An analysis on the field of data mining is done to show how data mining can help in business such as marketing, credit card approval. The project's objective is identifying the available data mining algorithms in data classification and applying new data mining algorithm to perform classification tasks. The proposed algorithm is a hybrid system which applied fuzzy logic and artificial neural network, which applies fuzzy logic inference to generate a set of fuzzy weighted production rules and applies artificial neural network to train the weights of fuzzy weighted rules for better classification results. Theresult of this system using the iris dataset and credit card approval dataset to evaluate the proposed algorithm's accuracy, interpretability. The project has achieved the target objectives; it can gain high accuracy for data classification task, generate rules which can help to interpret the output results, reduce the training processing. But the proposed algorithm still require high computation, the processing time will be long if the dataset is huge. However the proposed algorithm offers a promising approach to building intelligent systems

Data science for health-care: Patient condition recognition

>Magister Scientiae - MScThe emergence of the Internet of Things (IoT) and Artificial Intelligence (AI) have elicited increased interest in many areas of our daily lives. These include health, agriculture, aviation, manufacturing, cities management and many others. In the health sector, portable vital sign monitoring devices are being developed using the IoT technology to collect patients’ vital signs in real-time. The vital sign data acquired by wearable devices is quantitative and machine learning techniques can be applied to find hidden patterns in the dataset and help the medical practitioner with decision making. There are about 30000 diseases known to man and no human being can possibly remember all of them, their relations to other diseases, their symptoms and whether the symptoms exhibited by the patients are early warnings of a fatal disease. In light of this, Medical Decision Support Systems (MDSS) can provide assistance in making these crucial assessments. In most decision support systems factors a ect each other; they can be contradictory, competitive, and complementary. All these factors contribute to the overall decision and have di erent degrees of influence [85]. However, while there is more need for automated processes to improve the health-care sector, most of MDSS and the associated devices are still under clinical trials. This thesis revisits cyber physical health systems (CPHS) with the objective of designing and implementing a data analytics platform that provides patient condition monitoring services in terms of patient prioritisation and disease identification [1]. Di erent machine learning algorithms are investigated by the platform as potential candidate for achieving patient prioritisation. These include multiple linear regression, multiple logistic regression, classification and regression decision trees, single hidden layer neural networks and deep neural networks. Graph theory concepts are used to design and implement disease identification. The data analytics platform analyses data from biomedical sensors and other descriptive data provided by the patients (this can be recent data or historical data) stored in a cloud which can be private local health Information organisation (LHIO) or belonging to a regional health information organisation (RHIO). Users of the data analytics platform consisting of medical practitioners and patients are assumed to interact with the platform through cities’ pharmacies , rural E-Health kiosks end user applications

Uncertainty and Interpretability Studies in Soft Computing with an Application to Complex Manufacturing Systems

In systems modelling and control theory, the benefits of applying neural networks have been extensively studied. Particularly in manufacturing processes, such as the prediction of mechanical properties of heat treated steels. However, modern industrial processes usually involve large amounts of data and a range of non-linear effects and interactions that might hinder their model interpretation. For example, in steel manufacturing the understanding of complex mechanisms that lead to the mechanical properties which are generated by the heat treatment process is vital. This knowledge is not available via numerical models, therefore an experienced metallurgist estimates the model parameters to obtain the required properties. This human knowledge and perception sometimes can be imprecise leading to a kind of cognitive uncertainty such as vagueness and ambiguity when making decisions. In system classification, this may be translated into a system deficiency - for example, small input changes in system attributes may result in a sudden and inappropriate change for class assignation. In order to address this issue, practitioners and researches have developed systems that are functional equivalent to fuzzy systems and neural networks. Such systems provide a morphology that mimics the human ability of reasoning via the qualitative aspects of fuzzy information rather by its quantitative analysis. Furthermore, these models are able to learn from data sets and to describe the associated interactions and non-linearities in the data. However, in a like-manner to neural networks, a neural fuzzy system may suffer from a lost of interpretability and transparency when making decisions. This is mainly due to the application of adaptive approaches for its parameter identification. Since the RBF-NN can be treated as a fuzzy inference engine, this thesis presents several methodologies that quantify different types of uncertainty and its influence on the model interpretability and transparency of the RBF-NN during its parameter identification. Particularly, three kind of uncertainty sources in relation to the RBF-NN are studied, namely: entropy, fuzziness and ambiguity. First, a methodology based on Granular Computing (GrC), neutrosophic sets and the RBF-NN is presented. The objective of this methodology is to quantify the hesitation produced during the granular compression at the low level of interpretability of the RBF-NN via the use of neutrosophic sets. This study also aims to enhance the disitnguishability and hence the transparency of the initial fuzzy partition. The effectiveness of the proposed methodology is tested against a real case study for the prediction of the properties of heat-treated steels. Secondly, a new Interval Type-2 Radial Basis Function Neural Network (IT2-RBF-NN) is introduced as a new modelling framework. The IT2-RBF-NN takes advantage of the functional equivalence between FLSs of type-1 and the RBF-NN so as to construct an Interval Type-2 Fuzzy Logic System (IT2-FLS) that is able to deal with linguistic uncertainty and perceptions in the RBF-NN rule base. This gave raise to different combinations when optimising the IT2-RBF-NN parameters. Finally, a twofold study for uncertainty assessment at the high-level of interpretability of the RBF-NN is provided. On the one hand, the first study proposes a new methodology to quantify the a) fuzziness and the b) ambiguity at each RU, and during the formation of the rule base via the use of neutrosophic sets theory. The aim of this methodology is to calculate the associated fuzziness of each rule and then the ambiguity related to each normalised consequence of the fuzzy rules that result from the overlapping and to the choice with one-to-many decisions respectively. On the other hand, a second study proposes a new methodology to quantify the entropy and the fuzziness that come out from the redundancy phenomenon during the parameter identification. To conclude this work, the experimental results obtained through the application of the proposed methodologies for modelling two well-known benchmark data sets and for the prediction of mechanical properties of heat-treated steels conducted to publication of three articles in two peer-reviewed journals and one international conference

Automatic Assessment of Medication States of Patients with Parkinson’s Disease using Wearable Sensors

Motor fluctuations are a major focus of clinical managements in patients with mid-stage and advance Parkinson\u27s disease (PD). In this thesis, an automated algorithm is developed to identify those fluctuations (i.e., medication OFF and ON) using wearable sensors while PD patients are engaging in a variety of daily living activities. Four different methods are proposed which are supervised learning using Support Vector machine (SVM) with fuzzy classification, semi-supervised learning using k-means or using Self-organizing Tree Map Algorithm (SOTM) with fuzzy classification, and supervised classification using Long short-term memory (LSTM) as a deep learning method. A set of temporal and spectral features are extracted from the ambulatory signals of triaxial gyroscope sensors. After performing dimensionality reduction, the features are introduced to SVM or clustering methods using k-means or SOTM. Signals of the gyroscope sensors are passed directly to LSTM network. The developed methods were evaluated on two datasets that included recordings of 19 PD patients. Two scenarios were considered: general training/classification and patient-specific where the former trains and tests the algorithm using subject-based leave-one-out cross-validation, and the latter trains and tests the algorithm for each patient individually. In addition, for patient-specific scenarios, the number and placement of sensors is selected for each patient and this selection is based on the average change in UPDRS score between ON and OFF medication states and the presence of tremor for that patient. Overall, patient-specific algorithm resulted in a higher classification performance when it based on SVM with fuzzy classification (i.e., 80%, 82% and 78% for accuracy, sensitivity, and specificity of the OFF state, respectively). This algorithm was able to classify the medication states with high confidence (i.e., accuracy 94.86%, sensitivity 91.94% and specificity 96.83%) for the group of patients with a change of more than 15 in their UPDRS score between the OFF and ON medication states. This results are promising and thus this algorithm can be potentially used in routine clinical practice to improve the quality of this group of PD patients. In addition to these results, when only one sensor mounted on the ankle was used in the general training scenario, the algorithm based on LSTM performed better than SVM with 74.91%, 69.42%, and 80.55% for accuracy, sensitivity, and specificity, respectively. The promising results of LSTM show the potential outcome of developing deep learning methods in this field

Workshop on Fuzzy Control Systems and Space Station Applications

The Workshop on Fuzzy Control Systems and Space Station Applications was held on 14-15 Nov. 1990. The workshop was co-sponsored by McDonnell Douglas Space Systems Company and NASA Ames Research Center. Proceedings of the workshop are presented