Empirical Risk Minimization for Probabilistic Grammars: Sample Complexity and Hardness of Learning

Probabilistic grammars are generative statistical models that are useful for compositional and sequential structures. They are used ubiquitously in computational linguistics. We present a framework, reminiscent of structural risk minimization, for empirical risk minimization of probabilistic grammars using the log-loss. We derive sample complexity bounds in this framework that apply both to the supervised setting and the unsupervised setting. By making assumptions about the underlying distribution that are appropriate for natural language scenarios, we are able to derive distribution-dependent sample complexity bounds for probabilistic grammars. We also give simple algorithms for carrying out empirical risk minimization using this framework in both the supervised and unsupervised settings. In the unsupervised case, we show that the problem of minimizing empirical risk is NP-hard. We therefore suggest an approximate algorithm, similar to expectation-maximization, to minimize the empirical risk. Learning from data is central to contemporary computational linguistics. It is in common in such learning to estimate a model in a parametric family using the maximum likelihood principle. This principle applies in the supervised case (i.e., using annotate

A Bayesian analysis of neutron spin echo data on polymer coated gold nanoparticles in aqueous solutions

We present a neutron spin echo study (NSE) of the nanosecond dynamics of polyethylene glycol (PEG) functionalised nanosized gold particles dissolved in D$_2$O at two temperatures and two different PEG molecular weights. The analysis of the NSE data was performed by applying a Bayesian approach to the description of time correlation function decays in terms of exponential terms, recently proved to be theoretically rigorous. This approach, which addresses in a direct way the fundamental issue of model choice in any dynamical analysis, provides here a guide to the most statistically supported way to follow the decay of the Intermediate Scattering Functions I(Q, t) by basing on statistical grounds the choice of the number of terms required for the description of the nanosecond dynamics of the studied systems. Then, the presented analysis avoids from the start resorting to a pre-selected framework and can be considered as model free. By comparing the results of PEG coated nanoparticles with those obtained in PEG2000 solutions, we were able to disentangle the translational diffusion of the nanoparticles from the internal dynamics of the polymer grafted to them, and to show that the polymer corona relaxation follows a pure exponential decay in agreement with the behavior predicted by coarse grained molecular dynamics simulations and theoretical models. This methodology has one further advantage: in the presence of a complex dynamical scenario I(Q,t) is often described in terms of the Kohlrausch-Williams-Watts function that can implicitly represent a distribution of relaxation times. By choosing to describe the I(Q,t) as a sum of exponential functions and with the support of the Bayesian approach, we can explicitly determine when a finer-structure analysis of the dynamical complexity of the system exists according to the available data without the risk of overparametrisation

The Nasdaq crash of April 2000: Yet another example of log-periodicity in a speculative bubble ending in a crash

The Nasdaq Composite fell another $\approx 10$ on Friday the 14'th of April 2000 signaling the end of a remarkable speculative high-tech bubble starting in spring 1997. The closing of the Nasdaq Composite at 3321 corresponds to a total loss of over 35% since its all-time high of 5133 on the 10'th of March 2000. Similarities to the speculative bubble preceding the infamous crash of October 1929 are quite striking: the belief in what was coined a ``New Economy'' both in 1929 and presently made share-prices of companies with three digits price-earning ratios soar. Furthermore, we show that the largest draw downs of the Nasdaq are outliers with a confidence level better than 99% and that these two speculative bubbles, as well as others, both nicely fit into the quantitative framework proposed by the authors in a series of recent papers.Comment: 15 pages including 7 figures. Accepted in Eur. Phys. J. The revised version contains significant parametric and non-parametric statistical tests which establishes the outlier nature of the largest market events and provides an objective definition of a cras

SuperSpike: Supervised learning in multi-layer spiking neural networks

A vast majority of computation in the brain is performed by spiking neural networks. Despite the ubiquity of such spiking, we currently lack an understanding of how biological spiking neural circuits learn and compute in-vivo, as well as how we can instantiate such capabilities in artificial spiking circuits in-silico. Here we revisit the problem of supervised learning in temporally coding multi-layer spiking neural networks. First, by using a surrogate gradient approach, we derive SuperSpike, a nonlinear voltage-based three factor learning rule capable of training multi-layer networks of deterministic integrate-and-fire neurons to perform nonlinear computations on spatiotemporal spike patterns. Second, inspired by recent results on feedback alignment, we compare the performance of our learning rule under different credit assignment strategies for propagating output errors to hidden units. Specifically, we test uniform, symmetric and random feedback, finding that simpler tasks can be solved with any type of feedback, while more complex tasks require symmetric feedback. In summary, our results open the door to obtaining a better scientific understanding of learning and computation in spiking neural networks by advancing our ability to train them to solve nonlinear problems involving transformations between different spatiotemporal spike-time patterns

Recommender Systems

The ongoing rapid expansion of the Internet greatly increases the necessity of effective recommender systems for filtering the abundant information. Extensive research for recommender systems is conducted by a broad range of communities including social and computer scientists, physicists, and interdisciplinary researchers. Despite substantial theoretical and practical achievements, unification and comparison of different approaches are lacking, which impedes further advances. In this article, we review recent developments in recommender systems and discuss the major challenges. We compare and evaluate available algorithms and examine their roles in the future developments. In addition to algorithms, physical aspects are described to illustrate macroscopic behavior of recommender systems. Potential impacts and future directions are discussed. We emphasize that recommendation has a great scientific depth and combines diverse research fields which makes it of interests for physicists as well as interdisciplinary researchers.Comment: 97 pages, 20 figures (To appear in Physics Reports