Adaptive density estimation under dependence

Assume that $(X_t)_{t\in\Z}$ is a real valued time series admitting a common marginal density $f$ with respect to Lebesgue's measure. Donoho {\it et al.} (1996) propose a near-minimax method based on thresholding wavelets to estimate $f$ on a compact set in an independent and identically distributed setting. The aim of the present work is to extend these results to general weak dependent contexts. Weak dependence assumptions are expressed as decreasing bounds of covariance terms and are detailed for different examples. The threshold levels in estimators $\widehat f_n$ depend on weak dependence properties of the sequence $(X_t)_{t\in\Z}$ through the constant. If these properties are unknown, we propose cross-validation procedures to get new estimators. These procedures are illustrated via simulations of dynamical systems and non causal infinite moving averages. We also discuss the efficiency of our estimators with respect to the decrease of covariances bounds

Gradient descent for sparse rank-one matrix completion for crowd-sourced aggregation of sparsely interacting workers

We consider worker skill estimation for the singlecoin Dawid-Skene crowdsourcing model. In practice skill-estimation is challenging because worker assignments are sparse and irregular due to the arbitrary, and uncontrolled availability of workers. We formulate skill estimation as a rank-one correlation-matrix completion problem, where the observed components correspond to observed label correlation between workers. We show that the correlation matrix can be successfully recovered and skills identifiable if and only if the sampling matrix (observed components) is irreducible and aperiodic. We then propose an efficient gradient descent scheme and show that skill estimates converges to the desired global optima for such sampling matrices. Our proof is original and the results are surprising in light of the fact that even the weighted rank-one matrix factorization problem is NP hard in general. Next we derive sample complexity bounds for the noisy case in terms of spectral properties of the signless Laplacian of the sampling matrix. Our proposed scheme achieves state-of-art performance on a number of real-world datasets.Published versio

On Hilberg's Law and Its Links with Guiraud's Law

Hilberg (1990) supposed that finite-order excess entropy of a random human text is proportional to the square root of the text length. Assuming that Hilberg's hypothesis is true, we derive Guiraud's law, which states that the number of word types in a text is greater than proportional to the square root of the text length. Our derivation is based on some mathematical conjecture in coding theory and on several experiments suggesting that words can be defined approximately as the nonterminals of the shortest context-free grammar for the text. Such operational definition of words can be applied even to texts deprived of spaces, which do not allow for Mandelbrot's ``intermittent silence'' explanation of Zipf's and Guiraud's laws. In contrast to Mandelbrot's, our model assumes some probabilistic long-memory effects in human narration and might be capable of explaining Menzerath's law.Comment: To appear in Journal of Quantitative Linguistic

Control and estimation with limited information: a game-theoretic approach

Modern control systems can be viewed as interconnections of spatially distributed multiple subsystems, where the individual subsystems share their information with each other through an underlying network that inherently introduces limitations on information flow. Inherent limitations on the flow of information among individual subsystems may stem from structural constraints of the network and/or communication constraints of the network. Hence, in order to design optimal control and estimation mechanisms for modern control systems, we must answer the following two practical but important questions: (1) What are the fundamental communication limits to achieve a desired control performance and stability? (2) What are the approaches one has to adopt to design a decentralized controller for a complex system to deal with structural constraints? In this thesis, we consider four different problems within a game-theoretic framework to address the above questions. The first part of the thesis considers problems of control and estimation with limited communication, which correspond to question (1) above. We first consider the minimax estimation problem with intermittent observations. In this setting, the disturbance in the dynamical system as well as the sensor noise are controlled by adversaries, and the estimator receives the sensor measurements only sporadically, with availability governed by an independent and identically distributed (i.i.d.) Bernoulli process. This problem is cast in the thesis within the framework of stochastic zero-sum dynamic games. First, a corresponding stochastic minimax state estimator (SMSE) is obtained, along with an associated generalized stochastic Riccati equation (GSRE). Then, the asymptotic behavior of the estimation error in terms of the GSRE is analyzed. We obtain threshold-type conditions on the rate of intermittent observations and the disturbance attenuation parameter, above which 1) the expected value of the GSRE is bounded from below and above by deterministic quantities, and 2) the norm of the sequence generated by the GSRE converges weakly to a unique stationary distribution. We then study the minimax control problem over unreliable communication channels. The transmission of packets from the plant output sensors to the controller, and from the controller to the plant, are over sporadically failing channels governed by two independent i.i.d. Bernoulli processes. Two different scenarios for unreliable communication channels are considered. The first one is when the communication channel provides perfect acknowledgments of successful transmissions of control packets through a clean reverse channel, which is the TCP (Transmission Control Protocol), and the second one is when there is no acknowledgment, which is the UDP (User Datagram Protocol). Under both scenarios, the thesis obtains output feedback minimax controllers; it also identifies a set of explicit existence conditions in terms of the disturbance attenuation parameter and the communication channel loss rates, above which the corresponding minimax controller achieves the desired performance and stability. In the second part of the thesis, we consider two different large-scale optimization problems via mean field game theory, which address structural constraints in the complex system stated in question (2) above. We first consider two classes of mean field games. The first problem (P1) is one where each agent minimizes an exponentiated performance index, capturing risk-sensitive behavior, whereas in the second problem (P2) each agent minimizes a worst-case risk-neutral performance index, where a fictitious agent or an adversary enters each agent's state system. For both problems, a mean field system for the corresponding problem is constructed to arrive at a best estimate of the actual mean field behavior in various senses in the large population regime. In the finite population regime, we show that there exist epsilon-Nash equilibria for both P1 and P2, where the corresponding individual Nash strategies are decentralized as functions of the local state information. In both cases, the positive parameter epsilon can be taken to be arbitrarily small as the population size grows. Finally, we show that the Nash equilibria for P1 and P2 both feature robustness due to the risk-sensitive and worst-case behaviors of the agents. In the last main chapter of the thesis, we study mean field Stackelberg differential games. There is one leader and a large number, say N, of followers. The leader holds a dominating position in the game, where he first chooses and then announces his optimal strategy, to which the N followers respond by playing a Nash game. The followers are coupled with each other through the mean field term, and are strongly influenced by the leader's strategy. From the leader's perspective, he is coupled with the N followers through the mean field term. In this setting, we characterize an approximated stochastic mean field process of the followers governed by the leader's strategy, which leads to a decentralized epsilon-Nash-Stackelberg equilibrium. As a consequence of decentralization, we subsequently show that the positive parameter epsilon can be picked arbitrarily small when the number of followers is arbitrarily large. In the thesis, we also include several numerical computations and simulations, which illustrate the theoretical results