Temporal evolution of generalization during learning in linear networks

We study generalization in a simple framework of feedforward linear networks with n inputs and n outputs, trained from examples by gradient descent on the usual quadratic error function. We derive analytical results on the behavior of the validation function corresponding to the LMS error function calculated on a set of validation patterns. We show that the behavior of the validation function depends critically on the initial conditions and on the characteristics of the noise. Under certain simple assumptions, if the initial weights are sufficiently small, the validation function has a unique minimum corresponding to an optimal stopping time for training for which simple bounds can be calculated. There exists also situations where the validation function can have more complicated and somewhat unexpected behavior such as multiple local minima (at most n) of variable depth and long but finite plateau effects. Additional results and possible extensions are briefly discussed

Neural Networks for Fingerprint Recognition

After collecting a data base of fingerprint images, we design a neural network algorithm for fingerprint recognition. When presented with a pair of fingerprint images, the algorithm outputs an estimate of the probability that the two images originate from the same finger. In one experiment, the neural network is trained using a few hundred pairs of images and its performance is subsequently tested using several thousand pairs of images originated from a subset of the database corresponding to 20 individuals. The error rate currently achieved is less than 0.5%. Additional results, extensions, and possible applications are also briefly discussed

Hybrid modeling, HMM/NN architectures, and protein applications

We describe a hybrid modeling approach where the parameters of a model are calculated and modulated by another model, typically a neural network (NN), to avoid both overfitting and underfitting. We develop the approach for the case of Hidden Markov Models (HMMs), by deriving a class of hybrid HMM/NN architectures. These architectures can be trained with unified algorithms that blend HMM dynamic programming with NN backpropagation. In the case of complex data, mixtures of HMMs or modulated HMMs must be used. NNs can then be applied both to the parameters of each single HMM, and to the switching or modulation of the models, as a function of input or context. Hybrid HMM/NN architectures provide a flexible NN parameterization for the control of model structure and complexity. At the same time, they can capture distributions that, in practice, are inaccessible to single HMMs. The HMM/NN hybrid approach is tested, in its simplest form, by constructing a model of the immunoglobulin protein family. A hybrid model is trained, and a multiple alignment derived, with less than a fourth of the number of parameters used with previous single HMMs

Limit distributions for large P\'{o}lya urns

We consider a two-color P\'{o}lya urn in the case when a fixed number $S$ of balls is added at each step. Assume it is a large urn that is, the second eigenvalue $m$ of the replacement matrix satisfies $1/2<m/S\leq1$. After $n$ drawings, the composition vector has asymptotically a first deterministic term of order $n$ and a second random term of order $n^{m/S}$. The object of interest is the limit distribution of this random term. The method consists in embedding the discrete-time urn in continuous time, getting a two-type branching process. The dislocation equations associated with this process lead to a system of two differential equations satisfied by the Fourier transforms of the limit distributions. The resolution is carried out and it turns out that the Fourier transforms are explicitly related to Abelian integrals over the Fermat curve of degree $m$. The limit laws appear to constitute a new family of probability densities supported by the whole real line.Comment: Published in at http://dx.doi.org/10.1214/10-AAP696 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

An Inflation Forecasting Model for the Euro Area.

With the European economic integration, the understanding of inflation and inflationary pressures requires to analyse both the national level and the whole Euro area level. This is true in particular for the inflation forecasts that are carried out within the Eurosystem and published four times a year in the ECB Monthly Bulletin. For that purpose, the Banque de France is currently building tools for the Euro area in addition to those already in use for France. The present study puts forward a simple model of short-term developments (one year ahead) in inflation, as measured by the Harmonized Index of Consumer Prices (HICP) of the Euro area. This model does not take into account the feed-back effect of prices on activity, which should be considered in order to analyse medium-term price developments. It could hence be improved along these lines in the future. The model includes seven equations, explaining the total HICP of the Euro area and some of its sector-based sub-indexes (services, manufacturing sector, unprocessed food, processed food, energy and underlying inflation, defined as HICP inflation excluding unprocessed food and energy prices). It uses exogenous variables such as unit labour cost, import deflator, indicators of tightening in the labour market, or in the goods market, and indirect tax indicators. We have favoured an empirical approach rather than a strict compliance with theoretical models, paying particularly attention to the fit of the equations to the data. However, this model is able to provide relevant economic interpretations of recent price developments. Finally, we assess the forecasting performance of the model in traditional in-sample and out-of-sample rolling event evaluations. To do so, the forecasts were compared to the ones obtained from simple autoregressive equations, which are also commonly used to forecast short-term price developments. On the whole, the model provides more accurate forecasts than those provided by the autoregressive model, and a sector-based disaggregated approach outperforms a single equation to forecast total HICP. Part of this result may come from dummy variables that correspond to well identified shocks that improve both the econometric characteristics and forecast performance of the equations of our model.Inflation ; Economic Modelling ; Forecast.

Wealth effects: the French case.

This paper studies the relationship between consumption and wealth based on the concept of cointegration. The analysis focuses on French data over the 1987 - 2006 period. This relationship is expressed in two ways: in terms of Marginal Propensity to Consume out of wealth (MPC) and in terms of Elasticity of consumption to wealth. Three concepts of consumption are investigated: total households consumption expenditure, consumption excluding financial services and consumption excluding durable goods. Different estimators are also considered. Based on the MPC approach, when considered as permanent by households, an increase (decrease) in total wealth of one euro would lead to an increase (decrease) of 1 cent in total consumption. In terms of elasticity, an increase (de- crease) of 10% in wealth would imply also a relatively small impact of 0.8 to 1.1% on consumption depending on the concept of consumption considered. In most cases, the effect of a change in financial wealth is bigger than of a change in housing wealth. The results indicate that the wealth effects are smaller in France than in the UK and US but close to what is observed in Italy. In addition, any deviation of the variables from their common trends is corrected at first by adjustments in disposable income in line with what has been uncovered by studies on Germany and consistent with the "saving for the rainy days" approach of Campbell (1987). But our results contrast with the seminal study of Lettau and Ludvigson (2004) in the US where asset prices make the bulk of the adjustment.consumption, wealth effect, France.

Support and density of the limit mm-ary search trees distribution

The space requirements of an $m$-ary search tree satisfies a well-known phase transition: when $m\leq 26$, the second order asymptotics is Gaussian. When $m\geq 27$, it is not Gaussian any longer and a limit $W$ of a complex-valued martingale arises. We show that the distribution of $W$ has a square integrable density on the complex plane, that its support is the whole complex plane, and that it has finite exponential moments. The proofs are based on the study of the distributional equation W\egalLoi\sum_{k=1}^mV_k^{\lambda}W_k, where $V_1, ..., V_m$ are the spacings of $(m-1)$ independent random variables uniformly distributed on $[0,1]$, $W_1, ..., W_m$ are independent copies of W which are also independent of $(V_1, ..., V_m)$ and $\lambda$ is a complex number

Who has been affected, how and why? The spillover of the global financial crisis to Sub-Saharan Africa and ways to recovery

This paper first presents a comprehensive analysis of the significance of different transmission channels of the global economic and financial crisis to Sub-Saharan African countries. It then examines the repercussions of the crisis for the growth of gross domestic product (GDP) and its components; this is complemented by a study of the responses of monetary and fiscal authorities to the challenges posed by the crisis, both in regional terms and on the basis of selected country case studies. Finally, the paper highlights medium-term to long-term challenges for ensuring a sustainable recovery and for fostering resilience against potential future shocks.The authors find that the intensity of the impact of the crisis varies widely across countries, with a lack of export diversification apparently having been particularly conducive to its transmission. However, the analysis of the magnitude of the observed swings in macroeconomic variables also reveals that although they were large, they were not exceptional and are comparable to fl uctuations Sub-Saharan Africa has witnessed in the recent past. Furthermore, in a non-negligible number of instances the extent of the slowdown seems to have been determined by domestic factors as well. Particularly, policies and conditions prior to the global recession, rather than crisis contagion per se, appear decisively to have shaped the scope of possible responses in many cases.As a result, many of the policy lessons Sub- Saharan Africa might draw from the crisis do not involve radical deviation from the policies in place before. Efforts to improve the management of resource revenue for commodity-dependent countries, necessary reforms of the economic and business environment to enable a diversification of the export base, and further regional integration might help to alleviate possible future external shocks. Additionally, the crisis re-emphasises the need to back growth prospects by redefining sectoral priorities. JEL Classification: E52, E31, D84balance of payments, Global economic crisis, intern. spillover, Regional growth, Sub-Saharan Africa

Smoothing equations for large P\'olya urns

Consider a balanced non triangular two-color P\'olya-Eggenberger urn process, assumed to be large which means that the ratio sigma of the replacement matrix eigenvalues satisfies 1/2<sigma <1. The composition vector of both discrete time and continuous time models admits a drift which is carried by the principal direction of the replacement matrix. In the second principal direction, this random vector admits also an almost sure asymptotics and a real-valued limit random variable arises, named WDT in discrete time and WCT in continous time. The paper deals with the distributions of both W. Appearing as martingale limits, known to be nonnormal, these laws remain up to now rather mysterious. Exploiting the underlying tree structure of the urn process, we show that WDT and WCT are the unique solutions of two distributional systems in some suitable spaces of integrable probability measures. These systems are natural extensions of distributional equations that already appeared in famous algorithmical problems like Quicksort analysis. Existence and unicity of the solutions of the systems are obtained by means of contracting smoothing transforms. Via the equation systems, we find upperbounds for the moments of WDT and WCT and we show that the laws of WDT and WCT are moment-determined. We also prove that WDT is supported by the whole real line and admits a continuous density (WCT was already known to have a density, infinitely differentiable on R\{0} and not bounded at the origin)

What Has 100 Billion Dollars Worth of Debt Relief Done for Low- Income Countries?

Between 1989 and 2003, low-income countries received $100 billion in debt relief. The stated objectives for much of this debt relief have been to reduce debt overhang and to free up recipient government resources for development spending that would otherwise have been used for debt service. In this paper we empirically assess the extent to which debt relief has been successful in meeting these objectives, using a newly-constructed database measuring the present value of debt relief for 62 low-income countries. We find little evidence that debt relief has affected the level and composition of public spending in recipient countries. We also do not find evidence that debt relief has raised growth, investment rates or the quality of policies and institutions among recipient countries. Although we cannot rule out the possibility that our failure to find evidence of positive impacts of debt relief is due to a variety of data and statistical problems, the evidence reported here does suggest that some skepticism is in order regarding the likely benefits of further large-scale debt relief.Debt Relief, HIPC, Low-Income Countries, Debt