Learning without Recall by Random Walks on Directed Graphs

We consider a network of agents that aim to learn some unknown state of the world using private observations and exchange of beliefs. At each time, agents observe private signals generated based on the true unknown state. Each agent might not be able to distinguish the true state based only on her private observations. This occurs when some other states are observationally equivalent to the true state from the agent's perspective. To overcome this shortcoming, agents must communicate with each other to benefit from local observations. We propose a model where each agent selects one of her neighbors randomly at each time. Then, she refines her opinion using her private signal and the prior of that particular neighbor. The proposed rule can be thought of as a Bayesian agent who cannot recall the priors based on which other agents make inferences. This learning without recall approach preserves some aspects of the Bayesian inference while being computationally tractable. By establishing a correspondence with a random walk on the network graph, we prove that under the described protocol, agents learn the truth exponentially fast in the almost sure sense. The asymptotic rate is expressed as the sum of the relative entropies between the signal structures of every agent weighted by the stationary distribution of the random walk.Comment: 6 pages, To Appear in Conference on Decision and Control 201

Learning without Recall: A Case for Log-Linear Learning

We analyze a model of learning and belief formation in networks in which agents follow Bayes rule yet they do not recall their history of past observations and cannot reason about how other agents' beliefs are formed. They do so by making rational inferences about their observations which include a sequence of independent and identically distributed private signals as well as the beliefs of their neighboring agents at each time. Fully rational agents would successively apply Bayes rule to the entire history of observations. This leads to forebodingly complex inferences due to lack of knowledge about the global network structure that causes those observations. To address these complexities, we consider a Learning without Recall model, which in addition to providing a tractable framework for analyzing the behavior of rational agents in social networks, can also provide a behavioral foundation for the variety of non-Bayesian update rules in the literature. We present the implications of various choices for time-varying priors of such agents and how this choice affects learning and its rate.Comment: in 5th IFAC Workshop on Distributed Estimation and Control in Networked Systems, (NecSys 2015

Distributed Learning with Infinitely Many Hypotheses

We consider a distributed learning setup where a network of agents sequentially access realizations of a set of random variables with unknown distributions. The network objective is to find a parametrized distribution that best describes their joint observations in the sense of the Kullback-Leibler divergence. Apart from recent efforts in the literature, we analyze the case of countably many hypotheses and the case of a continuum of hypotheses. We provide non-asymptotic bounds for the concentration rate of the agents' beliefs around the correct hypothesis in terms of the number of agents, the network parameters, and the learning abilities of the agents. Additionally, we provide a novel motivation for a general set of distributed Non-Bayesian update rules as instances of the distributed stochastic mirror descent algorithm.Comment: Submitted to CDC201

Local non-Bayesian social learning with stubborn agents

We study a social learning model in which agents iteratively update their beliefs about the true state of the world using private signals and the beliefs of other agents in a non-Bayesian manner. Some agents are stubborn, meaning they attempt to convince others of an erroneous true state (modeling fake news). We show that while agents learn the true state on short timescales, they "forget" it and believe the erroneous state to be true on longer timescales. Using these results, we devise strategies for seeding stubborn agents so as to disrupt learning, which outperform intuitive heuristics and give novel insights regarding vulnerabilities in social learning

Learning And Decision Making In Groups

Many important real-world decision-making problems involve group interactions among individuals with purely informational interactions. Such situations arise for example in jury deliberations, expert committees, medical diagnoses, etc. We model the purely informational interactions of group members, where they receive private information and act based on that information while also observing other people\u27s beliefs or actions. In the first part of the thesis, we address the computations that a rational (Bayesian) decision-maker should undertake to realize her optimal actions, maximizing her expected utility given all available information at every decision epoch. We use an approach called iterated eliminations of infeasible signals (IEIS) to model the thinking process as well as the calculations of a Bayesian agent in a group decision scenario. Accordingly, as the Bayesian agent attempts to infer the true state of the world from her sequence of observations, she recursively refines her belief about the signals that other players could have observed and beliefs that they would have hold given the assumption that other players are also rational. We show that IEIS algorithm runs in exponential time; however, when the group structure is a partially ordered set the Bayesian calculations simplify and polynomial-time computation of the Bayesian recommendations is possible. We also analyze the computational complexity of the Bayesian belief formation in groups and show that it is NP-hard. We investigate the factors underlying this computational complexity and show how belief calculations simplify in special network structures or cases with strong inherent symmetries. We finally give insights about the statistical efficiency (optimality) of the beliefs and its relations to computational efficiency. In the second part, we propose the no-recall model of inference for heuristic decision-making that is rooted in the Bayes rule but avoids the complexities of rational inference in group interactions. Accordingly to this model, the group members behave rationally at the initiation of their interactions with each other; however, in the ensuing decision epochs, they rely on heuristics that replicate their experiences from the first stage and can be justified as optimal responses to simplified versions of their complex environments. We study the implications of the information structure, together with the properties of the probability distributions, which determine the structure of the so-called ``Bayesian heuristics\u27\u27 that the agents follow in this model. We also analyze the group decision outcomes in two classes of linear action updates and log-linear belief updates and show that many inefficiencies arise in group decisions as a result of repeated interactions between individuals, leading to overconfident beliefs as well as choice-shifts toward extreme actions. Nevertheless, balanced regular structures demonstrate a measure of efficiency in terms of aggregating the initial information of individuals. Finally, we extend this model to a case where agents are exposed to a stream of private data in addition to observing each other\u27s actions and analyze properties of learning and convergence under the no-recall framework

Structural Results and Applications for Perturbed Markov Chains

Each day, most of us interact with a myriad of networks: we search for information on the web, connect with friends on social media platforms, and power our homes using the electrical grid. Many of these interactions have improved our lives, but some have caused new societal issues - social media facilitating the rise of fake news, for example. The goal of this thesis is to advance our understanding of these systems, in hopes improving beneficial interactions with networks while reducing the harm of detrimental ones. Our primary contributions are threefold. First, we devise new algorithms for estimating Personalized PageRank (PPR), a measure of similarity between the nodes in a network used in applications like web search and recommendation systems. In contrast to most existing PPR estimators, our algorithms exploit local graph structure to reduce estimation complexity. We show the analysis of such algorithms is tractable for certain random graph models, and that the key insights obtained from these models hold empirically for real graphs. Our second contribution is to apply ideas from the PPR literature to two other problems. First, we show that PPR estimators can be adapted to the policy evaluation problem in reinforcement learning. More specifically, we devise policy evaluation algorithms inspired by existing PPR estimators that leverage certain side information to reduce the sample complexity of existing methods. Second, we use analytical ideas from the PPR literature to show that convergence behavior and robustness are intimately related for a certain class of Markov chains. Finally, we study social learning over networks as a model for the spread of fake news. For this model, we characterize the learning outcome in terms of a novel measure of the “density” of users spreading fake news. Using this characterization, we also devise optimal strategies for seeding fake news spreaders so as to disrupt learning. These strategies empirically outperform intuitive heuristics on real social networks (despite not being provably optimal for such graphs) and thus provide new insights regarding vulnerabilities in social learning. While the topics studied in this thesis are diverse, a unifying mathematical theme is that of perturbed Markov chains. This includes perturbations that yield useful interpretations in various applications, that provide algorithmic and analytical advantages, and that disrupt some underlying system or process. Throughout the thesis, the perturbed Markov chain theme guides our analysis and suggests more general methodologies.PHDElectrical Engineering: SystemsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/155213/1/dvial_1.pd