Application of Wilcoxon Norm for increased Outlier Insensitivity in Function Approximation Problems

In system theory, characterization and identification are fundamental problems. When the plant behavior is completely unknown, it may be characterized using certain model and then, its identification may be carried out with some artificial neural networks(ANN) (like multilayer perceptron(MLP) or functional link artificial neural network(FLANN) ) or Radial Basis Functions(RBF) using some learning rules such as the back propagation (BP) algorithm. They offer flexibility, adaptability and versatility, for the use of a variety of approaches to meet a specific goal, depending upon the circumstances and the requirements of the design specifications. The first aim of the present thesis is to provide a framework for the systematic design of adaptation laws for nonlinear system identification and channel equalization. While constructing an artificial neural network or a radial basis function neural network, the designer is often faced with the problem of choosing a network of the right size for the task. Using a smaller neural network decreases the cost of computation and increases generalization ability. However, a network which is too small may never solve the problem, while a larger network might be able to. Transmission bandwidth being one of the most precious resources in digital communication, Communication channels are usually modeled as band-limited linear finite impulse response (FIR) filters with low pass frequency response

Solving the linear interval tolerance problem for weight initialization of neural networks

Determining good initial conditions for an algorithm used to train a neural network is considered a parameter estimation problem dealing with uncertainty about the initial weights. Interval Analysis approaches model uncertainty in parameter estimation problems using intervals and formulating tolerance problems. Solving a tolerance problem is defining lower and upper bounds of the intervals so that the system functionality is guaranteed within predefined limits. The aim of this paper is to show how the problem of determining the initial weight intervals of a neural network can be defined in terms of solving a linear interval tolerance problem. The proposed Linear Interval Tolerance Approach copes with uncertainty about the initial weights without any previous knowledge or specific assumptions on the input data as required by approaches such as fuzzy sets or rough sets. The proposed method is tested on a number of well known benchmarks for neural networks trained with the back-propagation family of algorithms. Its efficiency is evaluated with regards to standard performance measures and the results obtained are compared against results of a number of well known and established initialization methods. These results provide credible evidence that the proposed method outperforms classical weight initialization methods

Neural network based image capture for 3D reconstruction

The aim of the thesis is to build a neural network, which is capable of choosing frames from a video, which have important information for building a 3D map of the depicted structure without losing the 3D map accuracy. Many times, consecutive frames have redundant information, which do not add to 3D map any significant information or some frames might be, for example, distorted, which do not add to 3D map at all. It all depends on how a camera is moved around when a video is filmed. If all the frames of the video are used in the reconstruction of the 3D map, it will take a long time and it will require a lot of resources, which is problematic especially in the embedded devices. In this thesis it has been considered that embedded device would choose the most informative frames for building the 3D map, but the 3D map itself would be built afterwards with the saved frames on a desktop computer. A database is built from video feeds for neural network training and testing. To build the data base for training a neural network a visual simultaneous localization and mapping algorithm is used to extract features, connecting points between frames and estimate the camera movement from each frame of the video feed. To get more training samples and make the training less time consuming, video feeds have been divided into short sequences of frames. A structure from motion algorithm is used to construct a 3D point cloud of image subsets. A 3D point cloud is then constructed after each frame. To determine whether a frame is a frame with important information for 3D point cloud construction, chamfer distance is used to calculate how close the 3D point cloud is after each added frame to the 3D point cloud constructed with all the video frames. Based on the chamfer distance change then class label is determined for each frame. For the neural network a long short-term memory recurrent neural network structure was chosen, because it can learn from the entire sequence of data. The data base construction, neural network training and validation all were done with Matlab. The result of this master’s thesis is a simple long short-term memory neural network that can choose the important frames from a short sequence of images, but the accuracy needs to be further improved to use the presented method in real embedded device. The custom loss function developed in the thesis did not perform well enough that any of the similar consecutive frames could be chosen, but not more than one of those.Diplomityön tarkoitus on rakentaa neuroverkko, joka pystyy valitsemaan tärkeät kuvat 3D-mallinnusta varten videosta ilman kuvauksen tarkkuuden heikentymistä verrattuna kuvaukseen, joka on tehty kaikilla videon kuvilla. Useasti peräkkäiset kuvat videossa sisältävät samanlaista tietoa, joka ei lisää 3D-mallinnukseen tarkkuutta. Kuinka paljon kuvissa on uutta tietoa verrattuna edelliseen kuvaan, riippuu kameran liikkeestä ja liikkeen nopeudesta. 3D-mallinnuksen rakentamiseen kuluu paljon aikaa ja laskentakapasiteettia, jos kaikkia videon kuvia käytetään 3D-mallinnuksen rakentamiseen, mikä on ongelmallista sulautetuissa järjestelmissä. Tässä työssä on käytetty oletusta, että sulautettu laite kuvaisi ympäristöä ja valitsisi kuvat, joissa on tärkeää informaatiota 3D kuvauksen tekemistä varten, jonka jälkeen valitut kuvat tallennettaisiin laitteen muistiin. Itse 3D-mallinnus tehtäisiin jälkikäteen pöytätietokoneella. Työssä on tehty tietokanta neuroverkkojen opetusta varten kokonaan pöytäkoneella. Tietokanta opetusta varten on tehty vSLAM-menetelmällä, jossa kuvista poimitaan piirteitä, joita voidaan yhdistää kuvien välillä ja niistä laskea kameran liike kuvien välillä. Jotta opetustietokantaa saadaan enemmän näytteitä, käytetyt videot on jaettu lyhyisiin kuvasarjoihin. Näin saadaan myös opetukseen käytettyä laskenta-aikaa lyhennettyä. SfM-menetelmällä on laskettu 3D-mallinnus kuvista, työssä on käytetty pistepilveä. Pistepilvet on laskettu jokaisen kuvan jälkeen. Kuva on määritelty tärkeäksi, jos sen lisääminen pistepilven laskentaan tekee pistepilvestä samanlaisemman viiste-etäisyydellä kuin pistepilvi, joka on laskettu kaikilla kuvasarjan kuvilla. Pistepilvien samanlaisuutta on mitattu viiste etäisyydellä jokaisen pistepilven laskentaan lisätyn kuvan jälkeen. Riippuen kuinka paljon viiste etäisyys pienenee kuvalle määritellään luokka. Neuroverkon rakenteena käytetään LSTM takaisinkytkeytyvää neuroverkkoa, koska se pystyy luokittelemaan jokaisen kuvan koko aikaisemman kuvajonon perusteella, eikä vain sen kuvan perusteella, jota parhaillaan luokitellaan. Matlab-ohjelmistoa on käytetty diplomityössä tietokannan ja neuroverkkojen rakentamiseen. Diplomityön tuloksena LTSM takaisinkytkeytyvä neuroverkko pystyy valitsemaan tärkeimpiä kuvia lyhyistä kuvasarjoista, mutta kuvien valintatarkkuutta pitää vielä tulevaisuudessa parantaa ennen kuin esitettyä järjestelmää voisi käyttää sulautetussa järjestelmässä. Neuroverkko ei oppinut valitsemaan yhtä ja vain yhtä kuvaa samanlaista tietoa sisältävien kuvien joukosta työssä käytetyillä riskifunktioilla

Statistical modelling by neural networks

In this thesis the two disciplines of Statistics and Artificial Neural Networks are combined into an integrated study of a data set of a weather modification Experiment. An extensive literature study on artificial neural network methodology has revealed the strongly interdisciplinary nature of the research and the applications in this field. An artificial neural networks are becoming increasingly popular with data analysts, statisticians are becoming more involved in the field. A recursive algoritlun is developed to optimize the number of hidden nodes in a feedforward artificial neural network to demonstrate how existing statistical techniques such as nonlinear regression and the likelihood-ratio test can be applied in innovative ways to develop and refine neural network methodology. This pruning algorithm is an original contribution to the field of artificial neural network methodology that simplifies the process of architecture selection, thereby reducing the number of training sessions that is needed to find a model that fits the data adequately. [n addition, a statistical model to classify weather modification data is developed using both a feedforward multilayer perceptron artificial neural network and a discriminant analysis. The two models are compared and the effectiveness of applying an artificial neural network model to a relatively small data set assessed. The formulation of the problem, the approach that has been followed to solve it and the novel modelling application all combine to make an original contribution to the interdisciplinary fields of Statistics and Artificial Neural Networks as well as to the discipline of meteorology.Mathematical SciencesD. Phil. (Statistics

Computing prime factorizations with neural networks

Master's Project (M.S.) University of Alaska Fairbanks, 2022When dealing with sufficiently large integers, even the most cutting-edge existing algorithms for computing prime factorizations are impractically slow. In this paper, we explore the possibility of using neural networks to approximate prime factorizations in the hopes of providing an alternative factorization method which trades accuracy for speed. Due to the intrinsic difficulty associated with this task, the focus of this paper is largely concentrated on the obstacles encountered in the training of the neural net, rather than on the viability of the method itself

Longitudinal clustering analysis and prediction of Parkinson\u27s disease progression using radiomics and hybrid machine learning

Background: We employed machine learning approaches to (I) determine distinct progression trajectories in Parkinson\u27s disease (PD) (unsupervised clustering task), and (II) predict progression trajectories (supervised prediction task), from early (years 0 and 1) data, making use of clinical and imaging features. Methods: We studied PD-subjects derived from longitudinal datasets (years 0, 1, 2 & 4; Parkinson\u27s Progressive Marker Initiative). We extracted and analyzed 981 features, including motor, non-motor, and radiomics features extracted for each region-of-interest (ROIs: left/right caudate and putamen) using our standardized standardized environment for radiomics analysis (SERA) radiomics software. Segmentation of ROIs on dopamine transposer - single photon emission computed tomography (DAT SPECT) images were performed via magnetic resonance images (MRI). After performing cross-sectional clustering on 885 subjects (original dataset) to identify disease subtypes, we identified optimal longitudinal trajectories using hybrid machine learning systems (HMLS), including principal component analysis (PCA) + K-Means algorithms (KMA) followed by Bayesian information criterion (BIC), Calinski-Harabatz criterion (CHC), and elbow criterion (EC). Subsequently, prediction of the identified trajectories from early year data was performed using multiple HMLSs including 16 Dimension Reduction Algorithms (DRA) and 10 classification algorithms. Results: We identified 3 distinct progression trajectories. Hotelling\u27s t squared test (HTST) showed that the identified trajectories were distinct. The trajectories included those with (I, II) disease escalation (2 trajectories, 27% and 38% of patients) and (III) stable disease (1 trajectory, 35% of patients). For trajectory prediction from early year data, HMLSs including the stochastic neighbor embedding algorithm (SNEA, as a DRA) as well as locally linear embedding algorithm (LLEA, as a DRA), linked with the new probabilistic neural network classifier (NPNNC, as a classifier), resulted in accuracies of 78.4% and 79.2% respectively, while other HMLSs such as SNEA + Lib_SVM (library for support vector machines) and t_SNE (t-distributed stochastic neighbor embedding) + NPNNC resulted in 76.5% and 76.1% respectively. Conclusions: This study moves beyond cross-sectional PD subtyping to clustering of longitudinal disease trajectories. We conclude that combining medical information with SPECT-based radiomics features, and optimal utilization of HMLSs, can identify distinct disease trajectories in PD patients, and enable effective prediction of disease trajectories from early year data

Prediction Interval Estimation Techniques for Empirical Modeling Strategies and their Applications to Signal Validation Tasks

The basis of this work was to evaluate both parametric and non-parametric empirical modeling strategies applied to signal validation or on-line monitoring tasks. On-line monitoring methods assess signal channel performance to aid in making instrument calibration decisions, enabling the use of condition-based calibration schedules. The three non-linear empirical modeling strategies studied were: artificial neural networks (ANN), neural network partial least squares (NNPLS), and local polynomial regression (LPR). These three types are the most common nonlinear models for applications to signal validation tasks. Of the class of local polynomials (for LPR), two were studied in this work: zero-order (kernel regression), and first-order (local linear regression). The evaluation of the empirical modeling strategies includes the presentation and derivation of prediction intervals for each of three different model types studied so that estimations could be made with an associated prediction interval. An estimate and its corresponding prediction interval contain the measurements with a specified certainty, usually 95%. The prediction interval estimates were compared to results obtained from bootstrapping via Monte Carlo resampling, to validate their expected accuracy. The estimation of prediction intervals applied to on-line monitoring systems is essential if widespread use of these empirical based systems is to be attained. In response to the topical report On-Line Monitoring of Instrument Channel Performance, published by the Electric Power Research Institute [Davis 1998], the NRC issued a safety evaluation report that identified the need to evaluate the associated uncertainty of empirical model estimations from all contributing sources. This need forms the basis for the research completed and reported in this dissertation. The focus of this work, and basis of its original contributions, were to provide an accurate prediction interval estimation method for each of the mentioned empirical modeling techniques, and to verify the results via bootstrap simulation studies. Properly determined prediction interval estimates were obtained that consistently captured the uncertainty of the given model such that the level of certainty of the intervals closely matched the observed level of coverage of the prediction intervals over the measured values. In most cases the expected level of coverage of the measured values within the prediction intervals was 95%, such that the probability that an estimate and its associated prediction interval contain the corresponding measured observation was 95%. The results also indicate that instrument channel drifts are identifiable through the use of the developed prediction intervals by observing the drop in the level of coverage of the prediction intervals to relatively low values, e.g. 30%. While all empirical models exhibit optimal performance for a given set of specifications, the identification of this optimal set may be difficult to attain. The developed methods of prediction interval estimation were shown to perform as expected over a wide range of model specifications, including misspecification. Model misspecification occurs through different mechanisms dependent on the type of empirical model. The main mechanisms under which model misspecification occur for each empirical model studied are: ANN – through architecture selection, NNPLS – through latent variable selection, LPR – through bandwidth selection. In addition, all of the above empirical models are susceptible to misspecification due to inadequate data and the presence of erroneous predictor variables in the set of predictors. A study was completed to verify that the presence of erroneous variables, i.e. unrelated to the desired response or random noise components, resulted in increases in the prediction interval magnitudes while maintaining the appropriate level of coverage for the response measurements. In addition to considering the resultant prediction intervals and coverage values, a comparative evaluation of the different empirical models was performed. The evaluation considers the average estimation errors and the stability of the models under repeated Monte Carlo resampling. The results indicate the large uncertainty of ANN models applied to collinear data, and the utility of the NNPLS model for the same purpose. While the results from the LPR models remained consistent for data with or without collinearity, assuming proper regularization was applied. The quantification of the uncertainty of an empirical model\u27s estimations is a necessary task for promoting the use of on-line monitoring systems in the nuclear power industry. All of the methods studied herein were applied to a simulated data set for an initial evaluation of the methods, and data from two different U.S. nuclear power plants for the purposes of signal validation for on-line monitoring tasks

Spatiotemporal Wireless Sensor Network Field Approximation with Multilayer Perceptron Artificial Neural Network Models

As sensors become increasingly compact and dependable in natural environments, spatially-distributed heterogeneous sensor network systems steadily become more pervasive. However, any environmental monitoring system must account for potential data loss due to a variety of natural and technological causes. Modeling a natural spatial region can be problematic due to spatial nonstationarities in environmental variables, and as particular regions may be subject to specific influences at different spatial scales. Relationships between processes within these regions are often ephemeral, so models designed to represent them cannot remain static. Integrating temporal factors into this model engenders further complexity. This dissertation evaluates the use of multilayer perceptron neural network models in the context of sensor networks as a possible solution to many of these problems given their data-driven nature, their representational flexibility and straightforward fitting process. The relative importance of parameters is determined via an adaptive backpropagation training process, which converges to a best-fit model for sensing platforms to validate collected data or approximate missing readings. As conditions evolve over time such that the model can no longer adapt to changes, new models are trained to replace the old. We demonstrate accuracy results for the MLP generally on par with those of spatial kriging, but able to integrate additional physical and temporal parameters, enabling its application to any region with a collection of available data streams. Potential uses of this model might be not only to approximate missing data in the sensor field, but also to flag potentially incorrect, unusual or atypical data returned by the sensor network. Given the potential for spatial heterogeneity in a monitored phenomenon, this dissertation further explores the benefits of partitioning a space and applying individual MLP models to these partitions. A system of neural models using both spatial and temporal parameters can be envisioned such that a spatiotemporal space partitioned by k-means is modeled by k neural models with internal weightings varying individually according to the dominant processes within the assigned region of each. Evaluated on simulated and real data on surface currents of theGulf ofMaine, partitioned models show significant improved results over single global models