103 research outputs found
Asymptotic law of likelihood ratio for multilayer perceptron models
We consider regression models involving multilayer perceptrons (MLP) with one
hidden layer and a Gaussian noise. The data are assumed to be generated by a
true MLP model and the estimation of the parameters of the MLP is done by
maximizing the likelihood of the model. When the number of hidden units of the
true model is known, the asymptotic distribution of the maximum likelihood
estimator (MLE) and the likelihood ratio (LR) statistic is easy to compute and
converge to a law. However, if the number of hidden unit is
over-estimated the Fischer information matrix of the model is singular and the
asymptotic behavior of the MLE is unknown. This paper deals with this case, and
gives the exact asymptotic law of the LR statistics. Namely, if the parameters
of the MLP lie in a suitable compact set, we show that the LR statistics is the
supremum of the square of a Gaussian process indexed by a class of limit score
functions.Comment: 19 page
Self Organizing Map algorithm and distortion measure
International audienceWe study the statistical meaning of the minimization of distortion measure and the relation between the equilibrium points of the SOM algorithm and the minima of distortion measure. If we assume that the observations and the map lie in an compact Euclidean space, we prove the strong consistency of the map which almost minimizes the empirical distortion. Moreover, after calculating the derivatives of the theoretical distortion measure, we show that the points minimizing this measure and the equilibria of the Kohonen map do not match in general. We illustrate, with a simple example, how this occurs
Asymptotics for regression models under loss of identifiability
This paper discusses the asymptotic behavior of regression models under general conditions. First, we give a general inequality for the difference of the sum of square errors (SSE) of the estimated regression model and the SSE of the theoretical best regression function in our model. A set of generalized derivative functions is a key tool in deriving such inequality. Under suitable Donsker condition for this set, we give the asymptotic distribution for the difference of SSE. We show how to get this Donsker property for parametric models even if the parameters characterizing the best regression function are not unique. This result is applied to neural networks regression models with redundant hidden units when loss of identifiability occurs
Estimating the Number of Regimes of Non-linear Autoregressive Models.
International audienceAutoregressive regime-switching models are being widely used in modelling financial and economic time series such as business cycles (Hamilton, 1989; Lam, 1990), exchange rates (Engle and Hamilton, 1990), financial panics (Schwert, 1989) or stock prices (Wong and Li, 2000). When the number of regimes is fixed the statistical inference is relatively straightforward and the asymptotic properties of the estimates may be established (Francq and Roussignol, 1998; Krishnamurthy and Rydén, 1998; Douc R., Moulines E. and Rydén T., 2004). However, the problem of selecting the number of regimes is far less obvious and hasn't been completely answered yet. When the number of regimes is unknown, identifiability problems arise and the likelihood ratio test statistic (LRTS hereafter) is no longer convergent to a -distribution. In this paper, we consider models which allow the series to switch between regimes and we propose to study such models without knowing the form of the density of the noise. The problem we address here is how to select the number of components or number of regimes. One possible method to answer this problem is to consider penalized criteria. The consistency of a modified BIC criterion was recently proven in the framework of likelihood criterion for linear switching models (see Oltéanu and Rynkiewicz). We extend these results to mixtures of nonlinear autoregressive models with mean square error criterion and prove the consistency of a penalized estimate for the number of components under some regularity conditions
Testing the number of parameters with multidimensional MLP
International audienceThis work concerns testing the number of parameters in one hidden layer multilayer perceptron (MLP). For this purpose we assume that we have identifiable models, up to a finite group of transformations on the weights, this is for example the case when the number of hidden units is know. In this framework, we show that we get a simple asymptotic distribution, if we use the logarithm of the determinant of the empirical error covariance matrix as cost function
Estimation and Test for Multidimensional Regression Models
This work is concerned with the estimation of multidimensional regression and
the asymptotic behaviour of the test involved in selecting models. The main
problem with such models is that we need to know the covariance matrix of the
noise to get an optimal estimator. We show in this paper that if we choose to
minimise the logarithm of the determinant of the empirical error covariance
matrix, then we get an asymptotically optimal estimator. Moreover, under
suitable assumptions, we show that this cost function leads to a very simple
asymptotic law for testing the number of parameters of an identifiable and
regular regression model. Numerical experiments confirm the theoretical
results
Consistance d'un estimateur de minimum de variance étendue
National audienceWe consider a generalization of the criterion minimized by the K-means algorithm, where a neighborhood structure is used in the calculus of the variance. Such tool is used, for example with Kohonen maps, to measure the quality of the quantification preserving the neighborhood relationships. If we assume that the parameter vector is in a compact Euclidean space and all it components are separated by a minimal distance, we show the strong consistency of the set of parameters almost realizing the minimum of the empirical extended variance
- …