An improved functional link neural network for data classification

The goal of classification is to assign the pre-specified group or class to an instance based on the observed features related to that instance. The implementation of several classification models is challenging as some only work well when the underlying assumptions are satisfied. In order to generate the complex mapping between input and output space to build the arbitrary complex non-linear decision boundaries, neural networks has become prominent tool with wide range of applications. The recent techniques such as Multilayer Perceptron (MLP), standard Functional Link Neural Network (FLNN) and Chebyshev Functional Link Neural Network (CFLNN) outperformed their existing regression, multiple regression, quadratic regression, stepwise polynomials, K-nearest neighbor (K-NN), Naïve Bayesian classifier and logistic regression. This research work explores the insufficiencies of well- known CFLNN model where CFLNN utilizes functional expansion with large number of degree and coefficient value for inputs enhancement which increase computational complexity of the network. Accordingly, two alternative models namely; Genocchi Functional Link Neural Network (GFLNN) and Chebyshev Wavelets Functional Link Neural Network (CWFLNN) are proposed. The novelty of these approaches is that, GFLNN presents the functional expansions with less degree and small coefficient values to make less computational inputs for training to overcome the drawbacks of CFLNN. Whereas, CWFLNN is capable to generate more number of small coefficient value based basis functions with same degree of polynomials as compared to other polynomials and it has orthonormality condition therefore it has more accurate constant of functional expansion and can approximate the functions within the interval. These properties of CWFLNN are used to overcome the deficiencies of GFLNN. The significance of proposed models is verified by using statistical tests such as Freidman test based on accuracy ranking and pairwise comparison test. Moreover, MLP, standard FLNN and CFLNN are used for comparison. For experiments, benched marked data sets from UCI repository, SVMLIB data set and KEEL data sets are utilized. The CWFLNN reveals significant improvement (due to its generating more numbers of basis function property) in terms of classification accuracy and reduces the computational work

Dense Associative Memory for Pattern Recognition

A model of associative memory is studied, which stores and reliably retrieves many more patterns than the number of neurons in the network. We propose a simple duality between this dense associative memory and neural networks commonly used in deep learning. On the associative memory side of this duality, a family of models that smoothly interpolates between two limiting cases can be constructed. One limit is referred to as the feature-matching mode of pattern recognition, and the other one as the prototype regime. On the deep learning side of the duality, this family corresponds to feedforward neural networks with one hidden layer and various activation functions, which transmit the activities of the visible neurons to the hidden layer. This family of activation functions includes logistics, rectified linear units, and rectified polynomials of higher degrees. The proposed duality makes it possible to apply energy-based intuition from associative memory to analyze computational properties of neural networks with unusual activation functions - the higher rectified polynomials which until now have not been used in deep learning. The utility of the dense memories is illustrated for two test cases: the logical gate XOR and the recognition of handwritten digits from the MNIST data set.Comment: Accepted for publication at NIPS 201

Sorting out typicality with the inverse moment matrix SOS polynomial

International audienceWe study a surprising phenomenon related to the representation of a cloud of data points using polynomials. We start with the previously unnoticed empirical observation that, given a collection (a cloud) of data points, the sublevel sets of a certain distinguished polynomial capture the shape of the cloud very accurately. This distinguished polynomial is a sum-of-squares (SOS) derived in a simple manner from the inverse of the empirical moment matrix. In fact, this SOS polynomial is directly related to orthogonal polynomials and the Christoffel function. This allows to generalize and interpret extremality properties of orthogonal polynomials and to provide a mathematical rationale for the observed phenomenon. Among diverse potential applications, we illustrate the relevance of our results on a network intrusion detection task for which we obtain performances similar to existing dedicated methods reported in the literature

Polar Varieties, Real Equation Solving and Data-Structures: The hypersurface case

In this paper we apply for the first time a new method for multivariate equation solving which was developed in \cite{gh1}, \cite{gh2}, \cite{gh3} for complex root determination to the {\em real} case. Our main result concerns the problem of finding at least one representative point for each connected component of a real compact and smooth hypersurface. The basic algorithm of \cite{gh1}, \cite{gh2}, \cite{gh3} yields a new method for symbolically solving zero-dimensional polynomial equation systems over the complex numbers. One feature of central importance of this algorithm is the use of a problem--adapted data type represented by the data structures arithmetic network and straight-line program (arithmetic circuit). The algorithm finds the complex solutions of any affine zero-dimensional equation system in non-uniform sequential time that is {\em polynomial} in the length of the input (given in straight--line program representation) and an adequately defined {\em geometric degree of the equation system}. Replacing the notion of geometric degree of the given polynomial equation system by a suitably defined {\em real (or complex) degree} of certain polar varieties associated to the input equation of the real hypersurface under consideration, we are able to find for each connected component of the hypersurface a representative point (this point will be given in a suitable encoding). The input equation is supposed to be given by a straight-line program and the (sequential time) complexity of the algorithm is polynomial in the input length and the degree of the polar varieties mentioned above.Comment: Late