Cause and Effect Prediction in Manufacturing Process Using an Improved Neural Networks

The limitations of the existing Knowledge Hyper-surface method in learning cause and effect relationships in the manufacturing process is explored. A new approach to enhance the performance of the current Knowledge Hyper-surface method has been proposed by constructing midpoints between each primary weight along each dimension by using a quadratic Lagrange interpolation polynomial. The new secondary-weight values, generated due to the addition of midpoints, were also represented as a linear combination of the corresponding primary/axial weight values. An improved neural networks in learning from examples have also been proposed where both of the proposed algorithms able to constrain the shape of the surface in two-dimensional and multi-dimensional cases and produced more realistic and acceptable results as compared to the previous version. The ability of the proposed approach to models the exponential increase/decrease in the belief values by using high-ordered polynomials without introducing ‘over-fitting’ effects was investigated. The performance of the proposed method in modelling the exponential increase/decrease in belief values was carried out on real cases taken from real casting data. The computed graphical results of the proposed methods were compared with the current Knowledge Hyper-surface and neural-network methods. As a result, the proposed methods correctly predict the sensitivity of process-parameter variations with the occurrence of a defect and very important area of research in a robust design methodology.

Identification of robotic manipulators' inverse dynamics coefficients via model-based adaptive networks

The values of a given manipulator's dynamics coefficients need to be accurately identified in order to employ model-based algorithms in the control of its motion. This thesis details the development of a novel form of adaptive network which is capable of accurately learning the coefficients of systems, such as manipulator inverse dynamics, where the algebraic form is known but the coefficients' values are not. Empirical motion data from a pair of PUMA 560s has been processed by the Context-Sensitive Linear Combiner (CSLC) network developed, and the coefficients of their inverse dynamics identified. The resultant precision of control is shown to be superior to that achieved from employing dynamics coefficients derived from direct measurement. As part of the development of the CSLC network, the process of network learning is examined. This analysis reveals that current network architectures for processing analogue output systems with high input order are highly unlikely to produce solutions that are good estimates throughout the entire problem space. In contrast, the CSLC network is shown to generalise intrinsically as a result of its structure, whilst its training is greatly simplified by the presence of only one minima in the network's error hypersurface. Furthermore, a fine-tuning algorithm for network training is presented which takes advantage of the CSLC network's single adaptive layer structure and does not rely upon gradient descent of the network error hypersurface, which commonly slows the later stages of network training

Neurons on Amoebae

We apply methods of machine-learning, such as neural networks, manifold learning and image processing, in order to study 2-dimensional amoebae in algebraic geometry and string theory. With the help of embedding manifold projection, we recover complicated conditions obtained from so-called lopsidedness. For certain cases it could even reach $\sim99\%$ accuracy, in particular for the lopsided amoeba of $F_0$ with positive coefficients which we place primary focus. Using weights and biases, we also find good approximations to determine the genus for an amoeba at lower computational cost. In general, the models could easily predict the genus with over $90\%$ accuracies. With similar techniques, we also investigate the membership problem, and image processing of the amoebae directly.Comment: 53 page

Optimal approximation of piecewise smooth functions using deep ReLU neural networks

We study the necessary and sufficient complexity of ReLU neural networks---in terms of depth and number of weights---which is required for approximating classifier functions in $L^2$. As a model class, we consider the set $\mathcal{E}^\beta (\mathbb R^d)$ of possibly discontinuous piecewise $C^\beta$ functions $f : [-1/2, 1/2]^d \to \mathbb R$, where the different smooth regions of $f$ are separated by $C^\beta$ hypersurfaces. For dimension $d \geq 2$, regularity $\beta > 0$, and accuracy $\varepsilon > 0$, we construct artificial neural networks with ReLU activation function that approximate functions from $\mathcal{E}^\beta(\mathbb R^d)$ up to $L^2$ error of $\varepsilon$. The constructed networks have a fixed number of layers, depending only on $d$ and $\beta$, and they have $O(\varepsilon^{-2(d-1)/\beta})$ many nonzero weights, which we prove to be optimal. In addition to the optimality in terms of the number of weights, we show that in order to achieve the optimal approximation rate, one needs ReLU networks of a certain depth. Precisely, for piecewise $C^\beta(\mathbb R^d)$ functions, this minimal depth is given---up to a multiplicative constant---by $\beta/d$. Up to a log factor, our constructed networks match this bound. This partly explains the benefits of depth for ReLU networks by showing that deep networks are necessary to achieve efficient approximation of (piecewise) smooth functions. Finally, we analyze approximation in high-dimensional spaces where the function $f$ to be approximated can be factorized into a smooth dimension reducing feature map $\tau$ and classifier function $g$---defined on a low-dimensional feature space---as $f = g \circ \tau$. We show that in this case the approximation rate depends only on the dimension of the feature space and not the input dimension.Comment: Generalized some estimates to $L^p$ norms for $0<p<\infty

Ab Initio Molecular Dynamics (Aimd)- a New Approach for Development of Accurate Potentials

In this study a new approach is presented for the development of accurate potential-energy hypersurfaces based on ab initio calculations that can be utilized to conduct molecular dynamics and Monte Carlo simulations to study chemical and mechanical properties at the atomistic level. The method integrates ab initio electronic structure calculations with the interpolation capability of multilayer neural networks. A sampling technique based on novelty detection is also developed to ensure that the neural network fitting for the potential energy spans the entire configuration space involved during the simulation. The procedure can be initiated using an empirical potential or direct dynamics simulation. The procedure is applied for developing the potential energy hypersurface for five-atom clusters within a silicon workpiece. Ab initio calculations were performed using Gaussian 98 electronic structure program. Results for five-atom silicon clusters representing the bulk and the surface structure are presented.Mechanical & Aerospace Engineerin

On the parameter combinations that matter and on those that do not: Data-driven studies of parameter (non)identifiability

We present a data-driven approach to characterizing nonidentifiability of a model’s parameters and illustrate it through dynamic as well as steady kinetic models. By employing Diffusion Maps and their extensions, we discover the minimal combinations of parameters required to characterize the output behavior of a chemical system: a set of effective parameters for the model. Furthermore, we introduce and use a Conformal Autoencoder Neural Network technique, as well as a kernel-based Jointly Smooth Function technique, to disentangle the redundant parameter combinations that do not affect the output behavior from the ones that do. We discuss the interpretability of our data-driven effective parameters, and demonstrate the utility of the approach both for behavior prediction and parameter estimation. In the latter task, it becomes important to describe level sets in parameter space that are consistent with a particular output behavior. We validate our approach on a model of multisite phosphorylation, where a reduced set of effective parameters (nonlinear combinations of the physical ones) has previously been established analytically

Towards a multipurpose neural network approach to novelty detection

Novelty detection, the identification of data that is unusual or different in some way, is relevant in a wide number of real-world scenarios, ranging from identifying unusual weather conditions to detecting evidence of damage in mechanical systems. However, utilising novelty detection approaches in a particular scenario presents significant challenges to the non-expert user. They must first select an appropriate approach from the novelty detection literature for their scenario. Then, suitable values must be determined for any parameters of the chosen approach. These challenges are at best time consuming and at worst prohibitively difficult for the user. Worse still, if no suitable approach can be found from the literature, then the user is left with the impossible task of designing a novelty detector themselves. In order to make novelty detection more accessible, an approach is required which does not pose the above challenges. This thesis presents such an approach, which aims to automatically construct novelty detectors for specific applications. The approach combines a neural network model, recently proposed to explain a phenomenon observed in the neural pathways of the retina, with an evolutionary algorithm that is capable of simultaneously evolving the structure and weights of a neural network in order to optimise its performance in a particular task. The proposed approach was evaluated over a number of very different novelty detection tasks. It was found that, in each task, the approach successfully evolved novelty detectors which outperformed a number of existing techniques from the literature. A number of drawbacks with the approach were also identified, and suggestions were given on ways in which these may potentially be overcome