Predictive Accuracy of Recommender Algorithms

Recommender systems present a customized list of items based upon user or item characteristics with the objective of reducing a large number of possible choices to a smaller ranked set most likely to appeal to the user. A variety of algorithms for recommender systems have been developed and refined including applications of deep learning neural networks. Recent research reports point to a need to perform carefully controlled experiments to gain insights about the relative accuracy of different recommender algorithms, because studies evaluating different methods have not used a common set of benchmark data sets, baseline models, and evaluation metrics. The dissertation used publicly available sources of ratings data with a suite of three conventional recommender algorithms and two deep learning (DL) algorithms in controlled experiments to assess their comparative accuracy. Results for the non-DL algorithms conformed well to published results and benchmarks. The two DL algorithms did not perform as well and illuminated known challenges implementing DL recommender algorithms as reported in the literature. Model overfitting is discussed as a potential explanation for the weaker performance of the DL algorithms and several regularization strategies are reviewed as possible approaches to improve predictive error. Findings justify the need for further research in the use of deep learning models for recommender systems

Comparison of Machine Learning Classifiers for Recognition of Online and Offline Handwritten Digits*

Automated recognition of handwritten digits has applications in several industries such as Postal and Banking for reading of addressed packages and cheques respectively. This paper compares four machine learning classifiers namely Naive Bayes, Instance Based Learner, Decision Tree and Neural Network for single digit recognition. Our experiments were conducted using the WEKA machine learning tool on two datasets; the MNIST offline handwritten digits and a collection of online ISGL handwritten digits acquired with a pen digitiser. Experiments were designed to allow for comparison within the datasets in a cross validation and across them where the online dataset is used for training and the offline dataset for testing and vice versa. We also compared classification accuracy at different levels of down sampling. Results indicate that the lazy learning instance based classifier performed slightly better than the neural network with a maximal accuracy of 97.86% and they both outperformed the other two classifiers: Naive Bayes and Decision Tree. The decision tree gave the worst performance of the four classifiers. We also discovered that better results were obtained with using the online digits when tested in a cross validation experiment. However, the pre-processed MNIST offline digits gave higher accuracies when used for training and tested with the online ISGL digits not vice versa. Also, we discovered down sampled size of 14x14 gave the best results for most of the four classifiers although these were not significantly different from the other down sampled sizes of 7x7, 21x21 and 28x28. We intend to investigate the performance of these classifiers in recognition of other characters (alphabets, punctuation and other symbols) as well as extend the recognition task to other levels of text granularity such as words, sentences and paragraphs. Keywords: Digits recognition, machine learning, classifiers, handwritten character recognition, Wek

Heart Diseases Diagnosis Using Artificial Neural Networks

Information technology has virtually altered every aspect of human life in the present era. The application of informatics in the health sector is rapidly gaining prominence and the benefits of this innovative paradigm are being realized across the globe. This evolution produced large number of patients’ data that can be employed by computer technologies and machine learning techniques, and turned into useful information and knowledge. This data can be used to develop expert systems to help in diagnosing some life-threating diseases such as heart diseases, with less cost, processing time and improved diagnosis accuracy. Even though, modern medicine is generating huge amount of data every day, little has been done to use this available data to solve challenges faced in the successful diagnosis of heart diseases. Highlighting the need for more research into the usage of robust data mining techniques to help health care professionals in the diagnosis of heart diseases and other debilitating disease conditions. Based on the foregoing, this thesis aims to develop a health informatics system for the classification of heart diseases using data mining techniques focusing on Radial Basis functions and emerging Neural Networks approach. The presented research involves three development stages; firstly, the development of a preliminary classification system for Coronary Artery Disease (CAD) using Radial Basis Function (RBF) neural networks. The research then deploys the deep learning approach to detect three different types of heart diseases i.e. Sleep Apnea, Arrhythmias and CAD by designing two novel classification systems; the first adopt a novel deep neural network method (with Rectified Linear unit activation) design as the second approach in this thesis and the other implements a novel multilayer kernel machine to mimic the behaviour of deep learning as the third approach. Additionally, this thesis uses a dataset obtained from patients, and employs normalization and feature extraction means to explore it in a unique way that facilitates its usage for training and validating different classification methods. This unique dataset is useful to researchers and practitioners working in heart disease treatment and diagnosis. The findings from the study reveal that the proposed models have high classification performance that is comparable, or perhaps exceed in some cases, the existing automated and manual methods of heart disease diagnosis. Besides, the proposed deep-learning models provide better performance when applied on large data sets (e.g., in the case of Sleep Apnea), with reasonable performance with smaller data sets. The proposed system for clinical diagnoses of heart diseases, contributes to the accurate detection of such disease, and could serve as an important tool in the area of clinic support system. The outcome of this study in form of implementation tool can be used by cardiologists to help them make more consistent diagnosis of heart diseases

Random Projection in Deep Neural Networks

This work investigates the ways in which deep learning methods can benefit from random projection (RP), a classic linear dimensionality reduction method. We focus on two areas where, as we have found, employing RP techniques can improve deep models: training neural networks on high-dimensional data and initialization of network parameters. Training deep neural networks (DNNs) on sparse, high-dimensional data with no exploitable structure implies a network architecture with an input layer that has a huge number of weights, which often makes training infeasible. We show that this problem can be solved by prepending the network with an input layer whose weights are initialized with an RP matrix. We propose several modifications to the network architecture and training regime that makes it possible to efficiently train DNNs with learnable RP layer on data with as many as tens of millions of input features and training examples. In comparison to the state-of-the-art methods, neural networks with RP layer achieve competitive performance or improve the results on several extremely high-dimensional real-world datasets. The second area where the application of RP techniques can be beneficial for training deep models is weight initialization. Setting the initial weights in DNNs to elements of various RP matrices enabled us to train residual deep networks to higher levels of performance

A review of artificial intelligence technologies in mineral identification : classification and visualization

Artificial intelligence is a branch of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine capable of responding in a manner similar to human intelligence. Research in this area includes robotics, language recognition, image identification, natural language processing, and expert systems. In recent years, the availability of large datasets, the development of effective algorithms, and access to powerful computers have led to unprecedented success in artificial intelligence. This powerful tool has been used in numerous scientific and engineering fields including mineral identification. This paper summarizes the methods and techniques of artificial intelligence applied to intelligent mineral identification based on research, classifying the methods and techniques as artificial neural networks, machine learning, and deep learning. On this basis, visualization analysis is conducted for mineral identification of artificial intelligence from field development paths, research hot spots, and keywords detection, respectively. In the end, based on trend analysis and keyword analysis, we propose possible future research directions for intelligent mineral identification.The National Natural Science Foundation of China.https://www.mdpi.com/journal/jsanElectrical, Electronic and Computer Engineerin

Intelligent Condition Monitoring and Prognostic Methods with Applications to Dynamic Seals in the Oil & Gas Industry

The capital-intensive oil & gas industry invests billions of dollars in equipment annually and it is important to keep the equipment in top operating condition to help maintain efficient process operations and improve the rate of return by predicting failures before incidents. Digitalization has taken over the world with advances in sensor technology, wireless communication and computational capabilities, however oil & gas industry has not taken full advantage of this despite being technology centric. Dynamic seals are a vital part of reciprocating and rotary equipment such as compressor, pumps, engines, etc. and are considered most frequently failing component. Polymeric seals are increasingly complex and non-linear in behavior and have been the research of interest since 1950s. Most of the prognostic studies on seals are physics-based and requires direct estimation of different physical parameters to assess the degradation of seals, which are often difficult to obtain during operation. Another feasible approach to predict the failure is from performance related sensor data and is termed as data-driven prognostics. The offline phase of this approach is where the performance related data from the component of interest are acquired, pre-processed and artificial intelligence tools or statistical methods are used to model the degradation of a system. The developed models are then deployed online for a real-time condition monitoring. There is a lack of research on the data-driven based tools and methods for dynamic seal prognosis. The primary goal in this dissertation is to develop offline data-driven intelligent condition monitoring and prognostic methods for two types of dynamic seals used in the oil & gas industry, to avoid fatal breakdown of rotary and reciprocating equipment. Accordingly, the interest in this dissertation lies in developing models to effectively evaluate and classify the running condition of rotary seals; assess the progression of degradation from its incipient to failure and to estimate the remaining useful life (RUL) of reciprocating seals. First, a data-driven prognostic framework is developed to classify the running condition of rotary seals. An accelerated aging and testing procedure simulating rotary seal operation in oil field is developed to capture the behavior of seals through their cycle of operation until failure. The diagnostic capability of torque, leakage and vibration signal in differentiating the health states of rotary seals using experiments are compared. Since the key features that differentiate the health condition of rotary seals are unknown, an extensive feature extraction in time and frequency domain is carried out and a wrapper-based feature selection approach is used to select relevant features, with Multilayer Perceptron neural network utilized as classification technique. The proposed approach has shown that features extracted from torque and leakage lack a better discriminating power on its own, in classifying the running condition of seals throughout its service life. The classifier built using optimal set of features from torque and leakage collectively has resulted in a high classification accuracy when compared to random forest and logistic regression, even for the data collected at a different operating condition. Second, a data-driven approach to predict the degradation process of reciprocating seals based on friction force signal using a hybrid Particle Swarm Optimization - Support Vector Machine is presented. There is little to no knowledge on the feature that reflects the degradation of reciprocating seals and on the application of SVM in predicting the future running condition of polymeric components such as seals. Controlled run-to-failure experiments are designed and performed, and data collected from a dedicated experimental set-up is used to develop the proposed approach. A degradation feature with high monotonicity is used as an indicator of seal degradation. The pseudo nearest neighbor is used to determine the essential number of inputs for forecasting the future trend. The most challenging aspect of tuning parameters in SVM is framed in terms of an optimization problem aimed at minimizing the prediction error. The results indicate the effectiveness and better accuracy of the proposed approach when compared to GA-SVM and XGBoost. Finally, a deep neural network-based approach for estimating remaining useful life of reciprocating seals, using force and leakage signals is presented. Time domain and frequency domain statistical features are extracted from the measurements. An ideal prognostic feature should be well correlated with degradation time, monotonically increasing or decreasing and robust to outliers. The identified metrics namely: monotonicity, correlation and robustness are used to evaluate the goodness of extracted features. Each of the three metric carries a relative importance in the RUL estimation and a weighted linear combination of the metrics are used to rank and select the best set of prognostic features. The redundancy in the selected features is eliminated using Kelley-Gardner-Sutcliffe penalty function-based correlation-clustering algorithm to select a representative feature from each of the clusters. Finally, RUL estimation is modeled using a deep neural network model. Run-to-failure data collected from a reciprocating set-up was used to validate this approach and the findings show that the proposed approach can improve the accuracy of RUL prediction when compared to PSO-SVM and XGBoost regression. This research has important contribution and implications to rotary and reciprocating seal domain in utilizing sensors along with machine learning algorithms in assessing the health state and prognosis of seals without any direct measurements. This research has paved the way to move from a traditional fail-and-fix to predict-and-prevent approach in maintenance of seals. The findings of this research are foundational for developing an online degradation assessment platform which can remotely monitor the performance degradation of seals and provide action recommendations on maintenance decisions. This would be of great interest to customers and oil field operators to improve equipment utilization, control maintenance cost by enabling just-in-time maintenance and increase rate of return on equipment by predicting failures before incidents