Machine learning: statistical physics based theory and smart industry applications

The increasing computational power and the availability of data have made it possible to train ever-bigger artificial neural networks. These so-called deep neural networks have been used for impressive applications, like advanced driver assistance and support in medical diagnoses. However, various vulnerabilities have been revealed and there are many open questions concerning the workings of neural networks. Theoretical analyses are therefore essential for further progress. One current question is: why is it that networks with Rectified Linear Unit (ReLU) activation seemingly perform better than networks with sigmoidal activation?We contribute to the answer to this question by comparing ReLU networks with sigmoidal networks in diverse theoretical learning scenarios. In contrast to analysing specific datasets, we use a theoretical modelling using methods from statistical physics. They give the typical learning behaviour for chosen model scenarios. We analyse both the learning behaviour on a fixed dataset and on a data stream in the presence of a changing task. The emphasis is on the analysis of the network’s transition to a state wherein specific concepts have been learnt. We find significant benefits of ReLU networks: they exhibit continuous increases of their performance and adapt more quickly to changing tasks.In the second part of the thesis we treat applications of machine learning: we design a quick quality control method for material in a production line and study the relationship with product faults. Furthermore, we introduce a methodology for the interpretable classification of time series data

Investigation into the effect of fixturing systems on the design of condition monitoring for machining operations

The global market competition has drawn the manufacturer’s attention on automated manufacturing processes using condition monitoring systems. These systems have been used for improving product quality, eliminating inspection, and enhancing manufacturing productivity. Fixtures are essential devices in machining processes to hold the tool or workpiece, hence they are influenced directly by the stability of the cutting tool. Therefore, tool and fixturing faults play an important part in the inaccuracy of the machining processes causing deterioration of surface roughness. For the above mentioned reasons, and the limited work in this domain, this thesis develops an experimental investigation to evaluate the effect of fixturing quality on the design of condition monitoring systems. The proposed monitoring system implements multisensors and signal processing methods able to analyse the sensory information and make an appropriate decision. Therefore, several sensors namely force, vibration, acoustic emission, eddy current, power, strain and sound, are combined with a newly suggested approach, named Taylor’s Equation Induced Pattern (TIP), and neural networks to detect tool wear and tool breakage. It also evaluates the monitoring system to provide valuable data to show the effect of fixturing quality. Surface roughness of the workpiece has been measured and compared with the sensitivity of the monitoring system, which reflects the state of tool and fixturing conditions

Development of soft computing and applications in agricultural and biological engineering

Soft computing is a set of “inexact” computing techniques, which are able to model and analyze very complex problems. For these complex problems, more conventional methods have not been able to produce cost-effective, analytical, or complete solutions. Soft computing has been extensively studied and applied in the last three decades for scientific research and engineering computing. In agricultural and biological engineering, researchers and engineers have developed methods of fuzzy logic, artificial neural networks, genetic algorithms, decision trees, and support vector machines to study soil and water regimes related to crop growth, analyze the operation of food processing, and support decision-making in precision farming. This paper reviews the development of soft computing techniques. With the concepts and methods, applications of soft computing in the field of agricultural and biological engineering are presented, especially in the soil and water context for crop management and decision support in precision agriculture. The future of development and application of soft computing in agricultural and biological engineering is discussed

Implementing decision tree-based algorithms in medical diagnostic decision support systems

As a branch of healthcare, medical diagnosis can be defined as finding the disease based on the signs and symptoms of the patient. To this end, the required information is gathered from different sources like physical examination, medical history and general information of the patient. Development of smart classification models for medical diagnosis is of great interest amongst the researchers. This is mainly owing to the fact that the machine learning and data mining algorithms are capable of detecting the hidden trends between features of a database. Hence, classifying the medical datasets using smart techniques paves the way to design more efficient medical diagnostic decision support systems. Several databases have been provided in the literature to investigate different aspects of diseases. As an alternative to the available diagnosis tools/methods, this research involves machine learning algorithms called Classification and Regression Tree (CART), Random Forest (RF) and Extremely Randomized Trees or Extra Trees (ET) for the development of classification models that can be implemented in computer-aided diagnosis systems. As a decision tree (DT), CART is fast to create, and it applies to both the quantitative and qualitative data. For classification problems, RF and ET employ a number of weak learners like CART to develop models for classification tasks. We employed Wisconsin Breast Cancer Database (WBCD), Z-Alizadeh Sani dataset for coronary artery disease (CAD) and the databanks gathered in Ghaem Hospital’s dermatology clinic for the response of patients having common and/or plantar warts to the cryotherapy and/or immunotherapy methods. To classify the breast cancer type based on the WBCD, the RF and ET methods were employed. It was found that the developed RF and ET models forecast the WBCD type with 100% accuracy in all cases. To choose the proper treatment approach for warts as well as the CAD diagnosis, the CART methodology was employed. The findings of the error analysis revealed that the proposed CART models for the applications of interest attain the highest precision and no literature model can rival it. The outcome of this study supports the idea that methods like CART, RF and ET not only improve the diagnosis precision, but also reduce the time and expense needed to reach a diagnosis. However, since these strategies are highly sensitive to the quality and quantity of the introduced data, more extensive databases with a greater number of independent parameters might be required for further practical implications of the developed models

Machine Learning

Machine Learning can be defined in various ways related to a scientific domain concerned with the design and development of theoretical and implementation tools that allow building systems with some Human Like intelligent behavior. Machine learning addresses more specifically the ability to improve automatically through experience

Framework for the Automation of SDLC Phases using Artificial Intelligence and Machine Learning Techniques

Software Engineering acts as a foundation stone for any software that is being built. It provides a common road-map for construction of software from any domain. Not following a well-defined Software Development Model have led to the failure of many software projects in the past. Agile is the Software Development Life Cycle (SDLC) Model that is widely used in practice in the IT industries to develop software on various technologies such as Big Data, Machine Learning, Artificial Intelligence, Deep learning. The focus on Software Engineering side in the recent years has been on trying to automate the various phases of SDLC namely- Requirements Analysis, Design, Coding, Testing and Operations and Maintenance. Incorporating latest trending technologies such as Machine Learning and Artificial Intelligence into various phases of SDLC, could facilitate for better execution of each of these phases. This in turn helps to cut-down costs, save time, improve the efficiency and reduce the manual effort required for each of these phases. The aim of this paper is to present a framework for the application of various Artificial Intelligence and Machine Learning techniques in the different phases of SDLC

Bacteria classification with an electronic nose employing artificial neural networks

This PhD thesis describes research for a medical application of electronic nose technology. There is a need at present for early detection of bacterial infection in order to improve treatment. At present, the clinical methods used to detect and classify bacteria types (usually using samples of infected matter taken from patients) can take up to two or three days. Many experienced medical staff, who treat bacterial infections, are able to recognise some types of bacteria from their odours. Identification of pathogens (i.e. bacteria responsible for disease) from their odours using an electronic nose could provide a rapid measurement and therefore early treatment. This research project used existing sensor technology in the form of an electronic nose in conjunction with data pre-processing and classification methods to classify up to four bacteria types from their odours. Research was performed mostly in the area of signal conditioning, data pre-processing and classification. A major area of interest was the use of artificial neural networks classifiers. There were three main objectives. First, to classify successfully a small range of bacteria types. Second, to identify issues relating to bacteria odour that affect the ability of an artificially intelligent system to classify bacteria from odour alone. And third, to establish optimal signal conditioning, data pre-processing and classification methods. The Electronic Nose consisted of a gas sensor array with temperature and humidity sensors, signal conditioning circuits, and gas flow apparatus. The bacteria odour was analysed using an automated sampling system, which used computer software to direct gas flow through one of several vessels (which were used to contain the odour samples, into the Electronic Nose. The electrical resistance of the odour sensors were monitored and output as electronic signals to a computer. The purpose of the automated sampling system was to improve repeatability and reduce human error. Further improvement of the Electronic Nose were implemented as a temperature control system which controlled the ambient gas temperature, and a new gas sensor chamber which incorporated improved gas flow. The odour data were collected and stored as numerical values within data files in the computer system. Once the data were stored in a non-volatile manner various classification experiments were performed. Comparisons were made and conclusions were drawn from the performance of various data pre-processing and classification methods. Classification methods employed included artificial neural networks, discriminant function analysis and multi-variate linear regression. For classifying one from four types, the best accuracy achieved was 92.78%. This was achieved using a growth phase compensated multiple layer perceptron. For identifying a single bacteria type from a mixture of two different types, the best accuracy was 96.30%. This was achieved using a standard multiple layer perceptron. Classification of bacteria odours is a typical `real world' application of the kind that electronic noses will have to be applied to if this technology is to be successful. The methods and principles researched here are one step towards the goal of introducing artificially intelligent sensor systems into everyday use. The results are promising and showed that it is feasible to used Electronic Nose technology in this application and that with further development useful products could be developed. The conclusion from this thesis is that an electronic nose can detect and classify different types of bacteria

Online Multi-Stage Deep Architectures for Feature Extraction and Object Recognition

Multi-stage visual architectures have recently found success in achieving high classification accuracies over image datasets with large variations in pose, lighting, and scale. Inspired by techniques currently at the forefront of deep learning, such architectures are typically composed of one or more layers of preprocessing, feature encoding, and pooling to extract features from raw images. Training these components traditionally relies on large sets of patches that are extracted from a potentially large image dataset. In this context, high-dimensional feature space representations are often helpful for obtaining the best classification performances and providing a higher degree of invariance to object transformations. Large datasets with high-dimensional features complicate the implementation of visual architectures in memory constrained environments. This dissertation constructs online learning replacements for the components within a multi-stage architecture and demonstrates that the proposed replacements (namely fuzzy competitive clustering, an incremental covariance estimator, and multi-layer neural network) can offer performance competitive with their offline batch counterparts while providing a reduced memory footprint. The online nature of this solution allows for the development of a method for adjusting parameters within the architecture via stochastic gradient descent. Testing over multiple datasets shows the potential benefits of this methodology when appropriate priors on the initial parameters are unknown. Alternatives to batch based decompositions for a whitening preprocessing stage which take advantage of natural image statistics and allow simple dictionary learners to work well in the problem domain are also explored. Expansions of the architecture using additional pooling statistics and multiple layers are presented and indicate that larger codebook sizes are not the only step forward to higher classification accuracies. Experimental results from these expansions further indicate the important role of sparsity and appropriate encodings within multi-stage visual feature extraction architectures