An improved functional link neural network for data classification

The goal of classification is to assign the pre-specified group or class to an instance based on the observed features related to that instance. The implementation of several classification models is challenging as some only work well when the underlying assumptions are satisfied. In order to generate the complex mapping between input and output space to build the arbitrary complex non-linear decision boundaries, neural networks has become prominent tool with wide range of applications. The recent techniques such as Multilayer Perceptron (MLP), standard Functional Link Neural Network (FLNN) and Chebyshev Functional Link Neural Network (CFLNN) outperformed their existing regression, multiple regression, quadratic regression, stepwise polynomials, K-nearest neighbor (K-NN), Naïve Bayesian classifier and logistic regression. This research work explores the insufficiencies of well- known CFLNN model where CFLNN utilizes functional expansion with large number of degree and coefficient value for inputs enhancement which increase computational complexity of the network. Accordingly, two alternative models namely; Genocchi Functional Link Neural Network (GFLNN) and Chebyshev Wavelets Functional Link Neural Network (CWFLNN) are proposed. The novelty of these approaches is that, GFLNN presents the functional expansions with less degree and small coefficient values to make less computational inputs for training to overcome the drawbacks of CFLNN. Whereas, CWFLNN is capable to generate more number of small coefficient value based basis functions with same degree of polynomials as compared to other polynomials and it has orthonormality condition therefore it has more accurate constant of functional expansion and can approximate the functions within the interval. These properties of CWFLNN are used to overcome the deficiencies of GFLNN. The significance of proposed models is verified by using statistical tests such as Freidman test based on accuracy ranking and pairwise comparison test. Moreover, MLP, standard FLNN and CFLNN are used for comparison. For experiments, benched marked data sets from UCI repository, SVMLIB data set and KEEL data sets are utilized. The CWFLNN reveals significant improvement (due to its generating more numbers of basis function property) in terms of classification accuracy and reduces the computational work

Different distance measures for fuzzy linear regression with Monte Carlo methods

The aim of this study was to determine the best distance measure for estimating the fuzzy linear regression model parameters with Monte Carlo (MC) methods. It is pointed out that only one distance measure is used for fuzzy linear regression with MC methods within the literature. Therefore, three different definitions of distance measure between two fuzzy numbers are introduced. Estimation accuracies of existing and proposed distance measures are explored with the simulation study. Distance measures are compared to each other in terms of estimation accuracy; hence this study demonstrates that the best distance measures to estimate fuzzy linear regression model parameters with MC methods are the distance measures defined by Kaufmann and Gupta (Introduction to fuzzy arithmetic theory and applications. Van Nostrand Reinhold, New York, 1991), Heilpern-2 (Fuzzy Sets Syst 91(2):259–268, 1997) and Chen and Hsieh (Aust J Intell Inf Process Syst 6(4):217–229, 2000). One the other hand, the worst distance measure is the distance measure used by Abdalla and Buckley (Soft Comput 11:991–996, 2007; Soft Comput 12:463–468, 2008). These results would be useful to enrich the studies that have already focused on fuzzy linear regression models

Estimating posterior distributions of synaptic strengths in a neural network using Approximate Bayesian Computation and Sequential Neural Posterior Estimation

In computational neuroscience, it is common to represent natural phenomena through mathematical models in the form of coupled differential equations. These models allow us to study neural networks in the brain by simulating and measuring neural activity, specifically spiking activity. This thesis examines a two-population neural network, consisting of one inhibitory and one excitatory population, to estimate the posterior distributions of four synaptic strength parameters. By varying these parameters in our simulations, we can analyze differences in spiking activity. From the spiking data, represented by histograms, we computed four summary statistics per population (eight in total): mean firing rate, Fano factor, mean interspike interval and coefficient of variation. We then construct three likelihood-free methods in order to create posterior distributions of the synaptic strength parameters based on either the summary statistics or the raw output data. The posteriors are constructed using Approximate Bayesian Computation (ABC) and Sequential Neural Posterior Estimation (SNPE) methods. One rejection-based ABC method, with included linear regression adjustment, is constructed, as well as two SNPE methods: one using the summary statistics and the other using an embedding network (consisting of a convolutional neural network) to extract it's own metrics based on the raw output data. Furthermore, we aim to evaluate the findings to best replicate the values of an observation. The observation is randomly selected from the simulated data, and then simulated an additional $100$ times with fixed parameter values to account for the stochastic properties of the network model. The mean of these additional simulations served as the observed summary statistics values, and the sample closest to this mean was selected as the raw output observation. To evaluate the posteriors, we examine the extent to which the methods are capable of restricting the synaptic strength parameters, comparing this with the observation's output in terms of both summary statistics and raw output data. The SNPE method using an embedding network is the least capable of restricting the parameter domain and replicating the observation results. The linear regression adjusted ABC method shows some improvement over this, while the SNPE method without an embedding network appears to be the most successful in both restricting the parameter domain and replicating the observation output

Possibilistic KNN regression using tolerance intervals

International audienceBy employing regression methods minimizing predictive risk, we are usually looking for precise values which tends to their true response value. However, in some situations, it may be more reasonable to predict intervals rather than precise values. In this paper, we focus to find such intervals for the K-nearest neighbors (KNN) method with precise values for inputs and output. In KNN, the prediction intervals are usually built by considering the local probability distribution of the neighborhood. In situations where we do not dispose of enough data in the neighborhood to obtain statistically significant distributions, we would rather wish to build intervals which takes into account such distribution uncertainties. For this latter we suggest to use tolerance intervals to build the maximal specific possibility distribution that bounds each population quantiles of the true distribution (with a fixed confidence level) that might have generated our sample set. Next we propose a new interval regression method based on KNN which take advantage of our possibility distribution in order to choose, for each instance, the value of K which will be a good trade-off between precision and uncertainty due to the limited sample size. Finally we apply our method on an aircraft trajectory prediction problem

Calibrated Prediction Intervals for Neural Network Regressors

Ongoing developments in neural network models are continually advancing the state of the art in terms of system accuracy. However, the predicted labels should not be regarded as the only core output; also important is a well-calibrated estimate of the prediction uncertainty. Such estimates and their calibration are critical in many practical applications. Despite their obvious aforementioned advantage in relation to accuracy, contemporary neural networks can, generally, be regarded as poorly calibrated and as such do not produce reliable output probability estimates. Further, while post-processing calibration solutions can be found in the relevant literature, these tend to be for systems performing classification. In this regard, we herein present two novel methods for acquiring calibrated predictions intervals for neural network regressors: empirical calibration and temperature scaling. In experiments using different regression tasks from the audio and computer vision domains, we find that both our proposed methods are indeed capable of producing calibrated prediction intervals for neural network regressors with any desired confidence level, a finding that is consistent across all datasets and neural network architectures we experimented with. In addition, we derive an additional practical recommendation for producing more accurate calibrated prediction intervals. We release the source code implementing our proposed methods for computing calibrated predicted intervals. The code for computing calibrated predicted intervals is publicly available

A Dynamic Approach to Linear Statistical Calibration with an Application in Microwave Radiometry

The problem of statistical calibration of a measuring instrument can be framed both in a statistical context as well as in an engineering context. In the first, the problem is dealt with by distinguishing between the 'classical' approach and the 'inverse' regression approach. Both of these models are static models and are used to estimate exact measurements from measurements that are affected by error. In the engineering context, the variables of interest are considered to be taken at the time at which you observe it. The Bayesian time series analysis method of Dynamic Linear Models (DLM) can be used to monitor the evolution of the measures, thus introducing an dynamic approach to statistical calibration. The research presented employs the use of Bayesian methodology to perform statistical calibration. The DLM's framework is used to capture the time-varying parameters that maybe changing or drifting over time. Two separate DLM based models are presented in this paper. A simulation study is conducted where the two models are compared to some well known 'static' calibration approaches in the literature from both the frequentist and Bayesian perspectives. The focus of the study is to understand how well the dynamic statistical calibration methods performs under various signal-to-noise ratios, r. The posterior distributions of the estimated calibration points as well as the 95% coverage intervals are compared by statistical summaries. These dynamic methods are applied to a microwave radiometry data set.Comment: 26 pages, 10 figure