On surrogate loss functions and ff-divergences

The goal of binary classification is to estimate a discriminant function $\gamma$ from observations of covariate vectors and corresponding binary labels. We consider an elaboration of this problem in which the covariates are not available directly but are transformed by a dimensionality-reducing quantizer $Q$. We present conditions on loss functions such that empirical risk minimization yields Bayes consistency when both the discriminant function and the quantizer are estimated. These conditions are stated in terms of a general correspondence between loss functions and a class of functionals known as Ali-Silvey or $f$-divergence functionals. Whereas this correspondence was established by Blackwell [Proc. 2nd Berkeley Symp. Probab. Statist. 1 (1951) 93--102. Univ. California Press, Berkeley] for the 0--1 loss, we extend the correspondence to the broader class of surrogate loss functions that play a key role in the general theory of Bayes consistency for binary classification. Our result makes it possible to pick out the (strict) subset of surrogate loss functions that yield Bayes consistency for joint estimation of the discriminant function and the quantizer.Comment: Published in at http://dx.doi.org/10.1214/08-AOS595 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

On the Design and Analysis of Secure Inference Networks

Parallel-topology inference networks consist of spatially-distributed sensing agents that collect and transmit observations to a central node called the fusion center (FC), so that a global inference is made regarding the phenomenon-of-interest (PoI). In this dissertation, we address two types of statistical inference, namely binary-hypothesis testing and scalar parameter estimation in parallel-topology inference networks. We address three different types of security threats in parallel-topology inference networks, namely Eavesdropping (Data-Confidentiality), Byzantine (Data-Integrity) or Jamming (Data-Availability) attacks. In an attempt to alleviate information leakage to the eavesdropper, we present optimal/near-optimal binary quantizers under two different frameworks, namely differential secrecy where the difference in performances between the FC and Eve is maximized, and constrained secrecy where FC’s performance is maximized in the presence of tolerable secrecy constraints. We also propose near-optimal transmit diversity mechanisms at the sensing agents in detection networks in the presence of tolerable secrecy constraints. In the context of distributed inference networks with M-ary quantized sensing data, we propose a novel Byzantine attack model and find optimal attack strategies that minimize KL Divergence at the FC in the presence of both ideal and non-ideal channels. Furthermore, we also propose a novel deviation-based reputation scheme to detect Byzantine nodes in a distributed inference network. Finally, we investigate optimal jamming attacks in detection networks where the jammer distributes its power across the sensing and the communication channels. We also model the interaction between the jammer and a centralized detection network as a complete information zero-sum game. We find closed-form expressions for pure-strategy Nash equilibria and show that both the players converge to these equilibria in a repeated game. Finally, we show that the jammer finds no incentive to employ pure-strategy equilibria, and causes greater impact on the network performance by employing mixed strategies

Rhythms of the Collective Brain: Metastable Synchronization and Cross-Scale Interactions in Connected Multitudes

Crowd behaviour challenges our fundamental understanding of social phenomena. Involving complex interactions between multiple temporal and spatial scales of activity, its governing mechanisms defy conventional analysis. Using 1.5 million Twitter messages from the 15M movement in Spain as an example of multitudinous self-organization, we describe the coordination dynamics of the system measuring phase-locking statistics at different frequencies using wavelet transforms, identifying 8 frequency bands of entrained oscillations between 15 geographical nodes. Then we apply maximum entropy inference methods to describe Ising models capturing transient synchrony in our data at each frequency band. The models show that (1) all frequency bands of the system operate near critical points of their parameter space and (2) while fast frequencies present only a few metastable states displaying all-or-none synchronization, slow frequencies present a diversity of metastable states of partial synchronization. Furthermore, describing the state at each frequency band using the energy of the corresponding Ising model, we compute transfer entropy to characterize cross-scale interactions between frequency bands, showing (1) a cascade of upward information flows in which each frequency band influences its contiguous slower bands and (2) downward information flows where slow frequencies modulate distant fast frequencies

Rhythms of the Collective Brain: Metastable Synchronization and Cross-Scale Interactions in Connected Multitudes

Crowd behaviour challenges our fundamental understanding of social phenomena. Involving complex interactions between multiple temporal and spatial scales of activity, its governing mechanisms defy conventional analysis. Using 1.5 million Twitter messages from the 15M movement in Spain as an example of multitudinous self-organization, we describe the coordination dynamics of the system measuring phase-locking statistics at different frequencies using wavelet transforms, identifying 8 frequency bands of entrained oscillations between 15 geographical nodes. Then we apply maximum entropy inference methods to describe Ising models capturing transient synchrony in our data at each frequency band. The models show that (1) all frequency bands of the system operate near critical points of their parameter space and (2) while fast frequencies present only a few metastable states displaying all-or-none synchronization, slow frequencies present a diversity of metastable states of partial synchronization. Furthermore, describing the state at each frequency band using the energy of the corresponding Ising model, we compute transfer entropy to characterize cross-scale interactions between frequency bands, showing (1) a cascade of upward information flows in which each frequency band influences its contiguous slower bands and (2) downward information flows where slow frequencies modulate distant fast frequencies