Multiscale network analysis through tail-greedy bottom-up approximation, with applications in neuroscience

We propose the TGUH (Tail-Greedy Unbalanced Haar) transform for networks, which results in an orthonormal, adaptive decomposition of the network adjacency matrix into Haar-wavelet like components. The `tail-greediness' of the algorithm - indicating multiple greedy steps are taken in a single pass through the data - enables both fast computation and consistent estimation of network signals. We focus on development of our multiscale network decomposition and a corresponding method for network signal denoising. Moreover, we establish consistency of our resulting denoising methodology, present numerical simulations illustrating compression, and illustrate through application to signals on diffusion tensor imaging (DTI) networks

Statistical methods for topology inference, denoising, and bootstrapping in networks

Quite often, the data we observe can be effectively represented using graphs. The underlying structure of the resulting graph, however, might contain noise and does not always hold constant across scales. With the right tools, we could possibly address these two problems. This thesis focuses on developing the right tools and provides insights in looking at them. Specifically, I study several problems that incorporate network data within the multi-scale framework, aiming at identifying common patterns and differences, of signals over networks across different scales. Additional topics in network denoising and network bootstrapping will also be discussed. The first problem we consider is the connectivity changes in dynamic networks constructed from multiple time series data. Multivariate time series data is often non-stationary. Furthermore, it is not uncommon to expect changes in a system across multiple time scales. Motivated by these observations, we in-corporate the traditional Granger-causal type of modeling within the multi-scale framework and propose a new method to detect the connectivity changes and recover the dynamic network structure. The second problem we consider is how to denoise and approximate signals over a network adjacency matrix. We propose an adaptive unbalanced Haar wavelet based transformation of the network data, and show that it is efficient in approximation and denoising of the graph signals over a network adjacency matrix. We focus on the exact decompositions of the network, the corresponding approximation theory, and denoising signals over graphs, particularly from the perspective of compression of the networks. We also provide a real data application on denoising EEG signals over a DTI network. The third problem we consider is in network denoising and network inference. Network representation is popular in characterizing complex systems. However, errors observed in the original measurements will propagate to network statistics and hence induce uncertainties to the summaries of the networks. We propose a spectral-denoising based resampling method to produce confidence intervals that propagate the inferential errors for a number of Lipschitz continuous net- work statistics. We illustrate the effectiveness of the method through a series of simulation studies

Detection and classification of non-stationary signals using sparse representations in adaptive dictionaries

Automatic classification of non-stationary radio frequency (RF) signals is of particular interest in persistent surveillance and remote sensing applications. Such signals are often acquired in noisy, cluttered environments, and may be characterized by complex or unknown analytical models, making feature extraction and classification difficult. This thesis proposes an adaptive classification approach for poorly characterized targets and backgrounds based on sparse representations in non-analytical dictionaries learned from data. Conventional analytical orthogonal dictionaries, e.g., Short Time Fourier and Wavelet Transforms, can be suboptimal for classification of non-stationary signals, as they provide a rigid tiling of the time-frequency space, and are not specifically designed for a particular signal class. They generally do not lead to sparse decompositions (i.e., with very few non-zero coefficients), and use in classification requires separate feature selection algorithms. Pursuit-type decompositions in analytical overcomplete (non-orthogonal) dictionaries yield sparse representations, by design, and work well for signals that are similar to the dictionary elements. The pursuit search, however, has a high computational cost, and the method can perform poorly in the presence of realistic noise and clutter. One such overcomplete analytical dictionary method is also analyzed in this thesis for comparative purposes. The main thrust of the thesis is learning discriminative RF dictionaries directly from data, without relying on analytical constraints or additional knowledge about the signal characteristics. A pursuit search is used over the learned dictionaries to generate sparse classification features in order to identify time windows that contain a target pulse. Two state-of-the-art dictionary learning methods are compared, the K-SVD algorithm and Hebbian learning, in terms of their classification performance as a function of dictionary training parameters. Additionally, a novel hybrid dictionary algorithm is introduced, demonstrating better performance and higher robustness to noise. The issue of dictionary dimensionality is explored and this thesis demonstrates that undercomplete learned dictionaries are suitable for non-stationary RF classification. Results on simulated data sets with varying background clutter and noise levels are presented. Lastly, unsupervised classification with undercomplete learned dictionaries is also demonstrated in satellite imagery analysis

Statistical Machine Learning Methods for High-dimensional Neural Population Data Analysis

Advances in techniques have been producing increasingly complex neural recordings, posing significant challenges for data analysis. This thesis discusses novel statistical methods for analyzing high-dimensional neural data. Part one discusses two extensions of state space models tailored to neural data analysis. First, we propose using a flexible count data distribution family in the observation model to faithfully capture over-dispersion and under-dispersion of the neural observations. Second, we incorporate nonlinear observation models into state space models to improve the flexibility of the model and get a more concise representation of the data. For both extensions, novel variational inference techniques are developed for model fitting, and simulated and real experiments show the advantages of our extensions. Part two discusses a fast region of interest (ROI) detection method for large-scale calcium imaging data based on structured matrix factorization. Part three discusses a method for sampling from a maximum entropy distribution with complicated constraints, which is useful for hypothesis testing for neural data analysis and many other applications related to maximum entropy formulation. We conclude the thesis with discussions and future works

On Neuroscience-Inspired Statistical and Computational Problems

Recent years have witnessed a surge of problems lying at the intersection of statistics andneuroscience. In this thesis, we explore various statistical and computational problems that are inspired by neuroscience. This thesis consists of two main parts, each inspired by a different system in the brain. In the first part, we study problems related to the visual system. In Chapter 2, we investigate the problem of estimating the collision time of a looming object using a theoretical formulation based on statistical hypothesis testing. In Chapter 3, we build computational models for the compound eye of Drosophila, and analyze how the models recover features of actual visual loom-selective neurons. In the second part, we study problems related to the memory system. In Chapter 4, we consider approaches for accelerating and reducing memory requirements for reinforcement learning algorithms, with provable guarantees on the performance of the algorithm

The Embedding Capacity of Information Flows Under Renewal Traffic

Given two independent point processes and a certain rule for matching points between them, what is the fraction of matched points over infinitely long streams? In many application contexts, e.g., secure networking, a meaningful matching rule is that of a maximum causal delay, and the problem is related to embedding a flow of packets in cover traffic such that no traffic analysis can detect it. We study the best undetectable embedding policy and the corresponding maximum flow rate ---that we call the embedding capacity--- under the assumption that the cover traffic can be modeled as arbitrary renewal processes. We find that computing the embedding capacity requires the inversion of very structured linear systems that, for a broad range of renewal models encountered in practice, admits a fully analytical expression in terms of the renewal function of the processes. Our main theoretical contribution is a simple closed form of such relationship. This result enables us to explore properties of the embedding capacity, obtaining closed-form solutions for selected distribution families and a suite of sufficient conditions on the capacity ordering. We evaluate our solution on real network traces, which shows a noticeable match for tight delay constraints. A gap between the predicted and the actual embedding capacities appears for looser constraints, and further investigation reveals that it is caused by inaccuracy of the renewal traffic model rather than of the solution itself.Comment: Sumbitted to IEEE Trans. on Information Theory on March 10, 201

Object Detection in 20 Years: A Survey

Object detection, as of one the most fundamental and challenging problems in computer vision, has received great attention in recent years. Its development in the past two decades can be regarded as an epitome of computer vision history. If we think of today's object detection as a technical aesthetics under the power of deep learning, then turning back the clock 20 years we would witness the wisdom of cold weapon era. This paper extensively reviews 400+ papers of object detection in the light of its technical evolution, spanning over a quarter-century's time (from the 1990s to 2019). A number of topics have been covered in this paper, including the milestone detectors in history, detection datasets, metrics, fundamental building blocks of the detection system, speed up techniques, and the recent state of the art detection methods. This paper also reviews some important detection applications, such as pedestrian detection, face detection, text detection, etc, and makes an in-deep analysis of their challenges as well as technical improvements in recent years.Comment: This work has been submitted to the IEEE TPAMI for possible publicatio

Adaptation of the Retina to Stimulus Correlations

Visual scenes in the natural world are highly correlated. To efficiently encode such an environment with a limited dynamic range, the retina ought to reduce correlations to maximize information. On the other hand, some redundancy is needed to combat the effects of noise. Here we ask how the degree of redundancy in retinal output depends on the stimulus ensemble. We find that retinal output preserves correlations in a spatially correlated stimulus but adaptively reduces changes in spatio-temporal input correlations. The latter effect can be explained by stimulus-dependent changes in receptive fields. We also find evidence that horizontal cells in the outer retina enhance changes in output correlations. GABAergic amacrine cells in the inner retina also enhance differences in correlation, albeit to a lesser degree, while gylcinergic amacrine cells have little effect on output correlation. These results suggest that the early visual system is capable of adapting to stimulus correlations to balance the challenges of redundancy and noise