Bootstrap methods for the empirical study of decision-making and information flows in social systems

Abstract: We characterize the statistical bootstrap for the estimation of information theoretic quantities from data, with particular reference to its use in the study of large-scale social phenomena. Our methods allow one to preserve, approximately, the underlying axiomatic relationships of information theory—in particular, consistency under arbitrary coarse-graining—that motivate use of these quantities in the first place, while providing reliability comparable to the state of the art for Bayesian estimators. We show how information-theoretic quantities allow for rigorous empirical study of the decision-making capacities of rational agents, and the time-asymmetric flows of information in distributed systems. We provide illustrative examples by reference to ongoing collaborative work on the semantic structure of the British Criminal Court system and the conflict dynamics of the contemporary Afghanistan insurgency

Decentralized Riemannian Particle Filtering with Applications to Multi-Agent Localization

The primary focus of this research is to develop consistent nonlinear decentralized particle filtering approaches to the problem of multiple agent localization. A key aspect in our development is the use of Riemannian geometry to exploit the inherently non-Euclidean characteristics that are typical when considering multiple agent localization scenarios. A decentralized formulation is considered due to the practical advantages it provides over centralized fusion architectures. Inspiration is taken from the relatively new field of information geometry and the more established research field of computer vision. Differential geometric tools such as manifolds, geodesics, tangent spaces, exponential, and logarithmic mappings are used extensively to describe probabilistic quantities. Numerous probabilistic parameterizations were identified, settling on the efficient square-root probability density function parameterization. The square-root parameterization has the benefit of allowing filter calculations to be carried out on the well studied Riemannian unit hypersphere. A key advantage for selecting the unit hypersphere is that it permits closed-form calculations, a characteristic that is not shared by current solution approaches. Through the use of the Riemannian geometry of the unit hypersphere, we are able to demonstrate the ability to produce estimates that are not overly optimistic. Results are presented that clearly show the ability of the proposed approaches to outperform current state-of-the-art decentralized particle filtering methods. In particular, results are presented that emphasize the achievable improvement in estimation error, estimator consistency, and required computational burden

Automated Segmentation of Left and Right Ventricles in MRI and Classification of the Myocarfium Abnormalities

A fundamental step in diagnosis of cardiovascular diseases, automated left and right ventricle (LV and RV) segmentation in cardiac magnetic resonance images (MRI) is still acknowledged to be a difficult problem. Although algorithms for LV segmentation do exist, they require either extensive training or intensive user inputs. RV segmentation in MRI has yet to be solved and is still acknowledged a completely unsolved problem because its shape is not symmetric and circular, its deformations are complex and varies extensively over the cardiac phases, and it includes papillary muscles. In this thesis, I investigate fast detection of the LV endo- and epi-cardium surfaces (3D) and contours (2D) in cardiac MRI via convex relaxation and distribution matching. A rapid 3D segmentation of the RV in cardiac MRI via distribution matching constraints on segment shape and appearance is also investigated. These algorithms only require a single subject for training and a very simple user input, which amounts to one click. The solution is sought following the optimization of functionals containing probability product kernel constraints on the distributions of intensity and geometric features. The formulations lead to challenging optimization problems, which are not directly amenable to convex-optimization techniques. For each functional, the problem is split into a sequence of sub-problems, each of which can be solved exactly and globally via a convex relaxation and the augmented Lagrangian method. Finally, an information-theoretic based artificial neural network (ANN) is proposed for normal/abnormal LV myocardium motion classification. Using the LV segmentation results, the LV cavity points is estimated via a Kalman filter and a recursive dynamic Bayesian filter. However, due to the similarities between the statistical information of normal and abnormal points, differentiating between distributions of abnormal and normal points is a challenging problem. The problem was investigated with a global measure based on the Shannon\u27s differential entropy (SDE) and further examined with two other information-theoretic criteria, one based on Renyi entropy and the other on Fisher information. Unlike the existing information-theoretic studies, the approach addresses explicitly the overlap between the distributions of normal and abnormal cases, thereby yielding a competitive performance. I further propose an algorithm based on a supervised 3-layer ANN to differentiate between the distributions farther. The ANN is trained and tested by five different information measures of radial distance and velocity for points on endocardial boundary

Unifying Consensus and Covariance Intersection for Efficient Distributed State Estimation over Unreliable Networks

This thesis studies the problem of recursive distributed state estimation over unreliable networks. The main contribution is to fuse the independent and dependent information separately. Local estimators communicate directly only with their immediate neighbors and nothing is assumed about the structure of the communication network, specifically it need not be connected at all times. The proposed estimator is a Hybrid one that fuses independent and dependent (or correlated) information using a distributed averaging and iterative conservative fusion rule respectively. It will be discussed how the hybrid method can improve estimators's performance and make it robust to network failures. The content of the thesis is divided in two main parts. In the first part I study how this idea is applied to the case of dynamical systems with continuous state and Gaussian noise. I establish bounds for estimation performance and show that my method produces unbiased conservative estimates that are better than Iterative Covariance Intersection (ICI). I will test the proposed algorithm on an atmospheric dispersion problem, a random linear system estimation and finally a target tracking problem. In the second part, I will discuss how the hybrid method can be applied to distributed estimation on a Hidden Markov Model. I will discuss the notion of conservativeness for general probability distributions and use the appropriate cost function to achieve improvement similar to the first part. The performance of the proposed method is evaluated in a multi-agent tracking problem and a high dimensional HMM and it is shown that its performance surpasses the competing algorithms

Exploiting Intrinsic Clustering Structure in Discrete-Valued Data Sets for Efficient Knowledge Discovery in the Presence of Missing Data

Scalable algorithm design has become central in the era of large-scale data analysis. The vast amounts of data pouring in from a diverse set of application domains, such as bioinformatics, recommender systems, sensor systems, and social networks, cannot be analyzed efficiently using many data mining and statistical tools that were designed for a small scale setting. It is an ongoing challenge to the data mining, machine learning, and statistics communities to design new methods for efficient data analysis. Confounding this challenge is the noisy and incomplete nature of real-world data sets. Research scientists as well as practitioners in industry need to find meaningful patterns in data with missing value rates often as high as 99%, in addition to errors in the data that can obstruct accurate analyses. My contribution to this line of research is the design of new algorithms for scalable clustering, data reduction, and similarity evaluation by exploiting inherent clustering structure in the input data to overcome the challenges of significant amounts of missing entries. I demonstrate that, by focusing on underlying clustering properties of the data, we can improve the efficiency of several data analysis methods on sparse, discrete-valued data sets. I highlight new methods that I have developed with my collaborators for three diverse knowledge discovery tasks: (1) clustering genetic markers into linkage groups, (2) reducing large-scale genetic data to a much smaller, more accurate representative data set, and (3) computing similarity between users in recommender systems. In each case, I point out how the underlying clustering structure can be used to design more efficient algorithms, even when high missing value rates are present

A Stochastic Resampling Based Selective Particle Filter for Robust Visual Object Tracking

In this work, a new variant of particle filter has been proposed. In visual object tracking, particle filters have been used popularly because they are compatible with system non-linearity and non-Gaussian posterior distribution. But the main problem in particle filtering is sample degeneracy. To solve this problem, a new variant of particle filter has been proposed. The resampling algorithm used in this proposed particle filter is derived by combining systematic resampling, which is commonly used in SIR-PF (Sampling Importance Resampling Particle Filter) and a modified bat algorithm; this resampling algorithm reduces sample degeneracy as well as sample impoverishments. The measurement model is modified to handle clutter in presence of varying background. A new motion dynamics model is proposed which further reduces the chance of sample degeneracy among the particles by adaptively shifting mean of the process noise. To deal with illumination fluctuation and object deformation in presence of complete occlusion, a template update algorithm has also been proposed. This template update algorithm can update template even when the difference in the spread of the color-histogram is especially large over time. The proposed tracker has been tested against many challenging conditions and found to be robust against clutter, illumination change, scale change, fast object movement, motion blur, and complete occlusion; it has been found that the proposed algorithm outperforms the SIR-PF (Sampling Importance Resampling Particle Filter), bat algorithm and some other state-of-the-art tracking algorithms

A precise bare simulation approach to the minimization of some distances. Foundations

In information theory -- as well as in the adjacent fields of statistics, machine learning, artificial intelligence, signal processing and pattern recognition -- many flexibilizations of the omnipresent Kullback-Leibler information distance (relative entropy) and of the closely related Shannon entropy have become frequently used tools. To tackle corresponding constrained minimization (respectively maximization) problems by a newly developed dimension-free bare (pure) simulation method, is the main goal of this paper. Almost no assumptions (like convexity) on the set of constraints are needed, within our discrete setup of arbitrary dimension, and our method is precise (i.e., converges in the limit). As a side effect, we also derive an innovative way of constructing new useful distances/divergences. To illustrate the core of our approach, we present numerous examples. The potential for widespread applicability is indicated, too; in particular, we deliver many recent references for uses of the involved distances/divergences and entropies in various different research fields (which may also serve as an interdisciplinary interface)

Satellite Cluster Tracking via Extent Estimation

Clusters of closely-spaced objects in orbit present unique tracking and prediction challenges. Association of observations to individual objects is often not possible until the objects have drifted sufficiently far apart from one another. This dissertation proposes a new paradigm for initial tracking of these clusters of objects: instead of tracking the objects independently, the cluster is tracked as a single entity, parameterized by its centroid and extent, or shape. The feasibility of this method is explored using a decoupled centroid and extent estimation scheme. The dynamics of the centroid of a cluster of satellites are studied, and a set of modified equinoctial elements is shown to minimize the discrepancy between the motion of the centroid and the observation-space centroid. The extent estimator is formulated as a matrix-variate particle filter. Several matrix similarity measures are tested as the filter weighting function, and the Bhattacharyya distance is shown to outperform the others in test cases. Finally, the combined centroid and extent filter is tested on a set of three on-orbit breakup events, generated using the NASA standard breakup model and simulated using realistic force models. The filter is shown to perform well across low-Earth, geosynchronous, and highly-elliptical orbits, with centroid error generally below five kilometers and well-fitting extent estimates. These results demonstrate that a decoupled centroid and extent filter can effectively track clusters of closely-spaced satellites. This could improve spaceflight safety by providing quantitative tracking information for the entire cluster much earlier than would otherwise be available through typical means