Interplay of Sensor Quantity, Placement and System Dimensionality on Energy Sparse Reconstruction of Fluid Flows

Reconstruction of fine-scale information from sparse data is relevant to many practical fluid dynamic applications where the sensing is typically sparse. Fluid flows in an ideal sense are manifestations of nonlinear multiscale PDE dynamical systems with inherent scale separation that impact the system dimensionality. There is a common need to analyze the data from flow measurements or high-fidelity computations for stability characteristics, identification of coherent structures and develop evolutionary models for real-time data-driven control. Given that sparse reconstruction is inherently an ill-posed problem, the most successful approaches require the knowledge of the underlying basis space spanning the manifold in which the system resides. In this study, we employ an approach that learns basis from singular value decomposition (SVD) of training data to reconstruct sparsely sensed information at randomly sampled locations. This allows us to leverage energy sparsity with l2 minimization instead of the more expensive, sparsity promoting l1 minimization. Further, for unknown flow systems where only global operating parameters such as Reynolds (Re) number and raw data are available, it is often not clear what the optimal number of sensors and their placement for near-exact reconstruction needs to be. In this effort, we explore the interplay of data sparsity, sparsity of the underlying flow system and sensor placement on energy sparse reconstruction performance enabled by data- driven SVD basis. To this end, we investigate sparse convolution-based reconstruction performance by characterizing operational bounds for canonical laminar cylinder wake flows in both limit-cycle and transient regimes.Comment: In considerations for publication in Inverse Problem

Local sparsity and recovery of fusion frames structured signals

The problem of recovering signals of high complexity from low quality sensing devices is analyzed via a combination of tools from signal processing and harmonic analysis. By using the rich structure offered by the recent development in fusion frames, we introduce a compressed sensing framework in which we split the dense information into sub-channel or local pieces and then fuse the local estimations. Each piece of information is measured by potentially low quality sensors, modeled by linear matrices and recovered via compressed sensing -- when necessary. Finally, by a fusion process within the fusion frames, we are able to recover accurately the original signal. Using our new method, we show, and illustrate on simple numerical examples, that it is possible, and sometimes necessary, to split a signal via local projections and / or filtering for accurate, stable, and robust estimation. In particular, we show that by increasing the size of the fusion frame, a certain robustness to noise can also be achieved. While the computational complexity remains relatively low, we achieve stronger recovery performance compared to usual single-device compressed sensing systems.Comment: 17 figures, 42 page

Blind Identification of Graph Filters

Network processes are often represented as signals defined on the vertices of a graph. To untangle the latent structure of such signals, one can view them as outputs of linear graph filters modeling underlying network dynamics. This paper deals with the problem of joint identification of a graph filter and its input signal, thus broadening the scope of classical blind deconvolution of temporal and spatial signals to the less-structured graph domain. Given a graph signal $\mathbf{y}$ modeled as the output of a graph filter, the goal is to recover the vector of filter coefficients $\mathbf{h}$, and the input signal $\mathbf{x}$ which is assumed to be sparse. While $\mathbf{y}$ is a bilinear function of $\mathbf{x}$ and $\mathbf{h}$, the filtered graph signal is also a linear combination of the entries of the lifted rank-one, row-sparse matrix $\mathbf{x} \mathbf{h}^T$. The blind graph-filter identification problem can thus be tackled via rank and sparsity minimization subject to linear constraints, an inverse problem amenable to convex relaxations offering provable recovery guarantees under simplifying assumptions. Numerical tests using both synthetic and real-world networks illustrate the merits of the proposed algorithms, as well as the benefits of leveraging multiple signals to aid the blind identification task

Robust flow field reconstruction from limited measurements via sparse representation

In many applications it is important to estimate a fluid flow field from limited and possibly corrupt measurements. Current methods in flow estimation often use least squares regression to reconstruct the flow field, finding the minimum-energy solution that is consistent with the measured data. However, this approach may be prone to overfitting and sensitive to noise. To address these challenges we instead seek a sparse representation of the data in a library of examples. Sparse representation has been widely used for image recognition and reconstruction, and it is well-suited to structured data with limited, corrupt measurements. We explore sparse representation for flow reconstruction on a variety of fluid data sets with a wide range of complexity, including vortex shedding past a cylinder at low Reynolds number, a mixing layer, and two geophysical flows. In addition, we compare several measurement strategies and consider various types of noise and corruption over a range of intensities. We find that sparse representation has considerably improved estimation accuracy and robustness to noise and corruption compared with least squares methods. We also introduce a sparse estimation procedure on local spatial patches for complex multiscale flows that preclude a global sparse representation. Based on these results, sparse representation is a promising framework for extracting useful information from complex flow fields with realistic measurements

Learning a Compressed Sensing Measurement Matrix via Gradient Unrolling

Linear encoding of sparse vectors is widely popular, but is commonly data-independent -- missing any possible extra (but a priori unknown) structure beyond sparsity. In this paper we present a new method to learn linear encoders that adapt to data, while still performing well with the widely used $\ell_1$ decoder. The convex $\ell_1$ decoder prevents gradient propagation as needed in standard gradient-based training. Our method is based on the insight that unrolling the convex decoder into $T$ projected subgradient steps can address this issue. Our method can be seen as a data-driven way to learn a compressed sensing measurement matrix. We compare the empirical performance of 10 algorithms over 6 sparse datasets (3 synthetic and 3 real). Our experiments show that there is indeed additional structure beyond sparsity in the real datasets; our method is able to discover it and exploit it to create excellent reconstructions with fewer measurements (by a factor of 1.1-3x) compared to the previous state-of-the-art methods. We illustrate an application of our method in learning label embeddings for extreme multi-label classification, and empirically show that our method is able to match or outperform the precision scores of SLEEC, which is one of the state-of-the-art embedding-based approaches.Comment: 17 pages, 7 tables, 8 figures, published in ICML 2019; part of this work was done while Shanshan was an intern at Google Research, New Yor

Dynamic Network Cartography

Communication networks have evolved from specialized, research and tactical transmission systems to large-scale and highly complex interconnections of intelligent devices, increasingly becoming more commercial, consumer-oriented, and heterogeneous. Propelled by emergent social networking services and high-definition streaming platforms, network traffic has grown explosively thanks to the advances in processing speed and storage capacity of state-of-the-art communication technologies. As "netizens" demand a seamless networking experience that entails not only higher speeds, but also resilience and robustness to failures and malicious cyber-attacks, ample opportunities for signal processing (SP) research arise. The vision is for ubiquitous smart network devices to enable data-driven statistical learning algorithms for distributed, robust, and online network operation and management, adaptable to the dynamically-evolving network landscape with minimal need for human intervention. The present paper aims at delineating the analytical background and the relevance of SP tools to dynamic network monitoring, introducing the SP readership to the concept of dynamic network cartography -- a framework to construct maps of the dynamic network state in an efficient and scalable manner tailored to large-scale heterogeneous networks.Comment: To appear in the IEEE Signal Processing Magazine - Special Issue on Adaptation and Learning over Complex Network

Task-Driven Dictionary Learning for Hyperspectral Image Classification with Structured Sparsity Constraints

Sparse representation models a signal as a linear combination of a small number of dictionary atoms. As a generative model, it requires the dictionary to be highly redundant in order to ensure both a stable high sparsity level and a low reconstruction error for the signal. However, in practice, this requirement is usually impaired by the lack of labelled training samples. Fortunately, previous research has shown that the requirement for a redundant dictionary can be less rigorous if simultaneous sparse approximation is employed, which can be carried out by enforcing various structured sparsity constraints on the sparse codes of the neighboring pixels. In addition, numerous works have shown that applying a variety of dictionary learning methods for the sparse representation model can also improve the classification performance. In this paper, we highlight the task-driven dictionary learning algorithm, which is a general framework for the supervised dictionary learning method. We propose to enforce structured sparsity priors on the task-driven dictionary learning method in order to improve the performance of the hyperspectral classification. Our approach is able to benefit from both the advantages of the simultaneous sparse representation and those of the supervised dictionary learning. We enforce two different structured sparsity priors, the joint and Laplacian sparsity, on the task-driven dictionary learning method and provide the details of the corresponding optimization algorithms. Experiments on numerous popular hyperspectral images demonstrate that the classification performance of our approach is superior to sparse representation classifier with structured priors or the task-driven dictionary learning method

Multi-View Task-Driven Recognition in Visual Sensor Networks

Nowadays, distributed smart cameras are deployed for a wide set of tasks in several application scenarios, ranging from object recognition, image retrieval, and forensic applications. Due to limited bandwidth in distributed systems, efficient coding of local visual features has in fact been an active topic of research. In this paper, we propose a novel approach to obtain a compact representation of high-dimensional visual data using sensor fusion techniques. We convert the problem of visual analysis in resource-limited scenarios to a multi-view representation learning, and we show that the key to finding properly compressed representation is to exploit the position of cameras with respect to each other as a norm-based regularization in the particular signal representation of sparse coding. Learning the representation of each camera is viewed as an individual task and a multi-task learning with joint sparsity for all nodes is employed. The proposed representation learning scheme is referred to as the multi-view task-driven learning for visual sensor network (MT-VSN). We demonstrate that MT-VSN outperforms state-of-the-art in various surveillance recognition tasks.Comment: 5 pages, Accepted in International Conference of Image Processing, 201

Sparse Time-Frequency decomposition by dictionary learning

In this paper, we propose a time-frequency analysis method to obtain instantaneous frequencies and the corresponding decomposition by solving an optimization problem. In this optimization problem, the basis to decompose the signal is not known. Instead, it is adapted to the signal and is determined as part of the optimization problem. In this sense, this optimization problem can be seen as a dictionary learning problem. This dictionary learning problem is solved by using the Augmented Lagrangian Multiplier method (ALM) iteratively. We further accelerate the convergence of the ALM method in each iteration by using the fast wavelet transform. We apply our method to decompose several signals, including signals with poor scale separation, signals with outliers and polluted by noise and a real signal. The results show that this method can give accurate recovery of both the instantaneous frequencies and the intrinsic mode functions

Application of Compressive Sensing Techniques in Distributed Sensor Networks: A Survey

In this survey paper, our goal is to discuss recent advances of compressive sensing (CS) based solutions in wireless sensor networks (WSNs) including the main ongoing/recent research efforts, challenges and research trends in this area. In WSNs, CS based techniques are well motivated by not only the sparsity prior observed in different forms but also by the requirement of efficient in-network processing in terms of transmit power and communication bandwidth even with nonsparse signals. In order to apply CS in a variety of WSN applications efficiently, there are several factors to be considered beyond the standard CS framework. We start the discussion with a brief introduction to the theory of CS and then describe the motivational factors behind the potential use of CS in WSN applications. Then, we identify three main areas along which the standard CS framework is extended so that CS can be efficiently applied to solve a variety of problems specific to WSNs. In particular, we emphasize on the significance of extending the CS framework to (i). take communication constraints into account while designing projection matrices and reconstruction algorithms for signal reconstruction in centralized as well in decentralized settings, (ii) solve a variety of inference problems such as detection, classification and parameter estimation, with compressed data without signal reconstruction and (iii) take practical communication aspects such as measurement quantization, physical layer secrecy constraints, and imperfect channel conditions into account. Finally, open research issues and challenges are discussed in order to provide perspectives for future research directions