One-Bit Quantization Design and Adaptive Methods for Compressed Sensing

There have been a number of studies on sparse signal recovery from one-bit quantized measurements. Nevertheless, little attention has been paid to the choice of the quantization thresholds and its impact on the signal recovery performance. This paper examines the problem of one-bit quantizer design for sparse signal recovery. Our analysis shows that the magnitude ambiguity that ever plagues conventional one-bit compressed sensing methods can be resolved, and an arbitrarily small reconstruction error can be achieved by setting the quantization thresholds close enough to the original data samples without being quantized. Note that unquantized data samples are unaccessible in practice. To overcome this difficulty, we propose an adaptive quantization method that adaptively adjusts the quantization thresholds in a way such that the thresholds converges to the optimal thresholds. Numerical results are illustrated to collaborate our theoretical results and the effectiveness of the proposed algorithm

Fast Asynchronous Parallel Stochastic Gradient Decent

Stochastic gradient descent~(SGD) and its variants have become more and more popular in machine learning due to their efficiency and effectiveness. To handle large-scale problems, researchers have recently proposed several parallel SGD methods for multicore systems. However, existing parallel SGD methods cannot achieve satisfactory performance in real applications. In this paper, we propose a fast asynchronous parallel SGD method, called AsySVRG, by designing an asynchronous strategy to parallelize the recently proposed SGD variant called stochastic variance reduced gradient~(SVRG). Both theoretical and empirical results show that AsySVRG can outperform existing state-of-the-art parallel SGD methods like Hogwild! in terms of convergence rate and computation cost

Super-Resolution Compressed Sensing: An Iterative Reweighted Algorithm for Joint Parameter Learning and Sparse Signal Recovery

In many practical applications such as direction-of-arrival (DOA) estimation and line spectral estimation, the sparsifying dictionary is usually characterized by a set of unknown parameters in a continuous domain. To apply the conventional compressed sensing to such applications, the continuous parameter space has to be discretized to a finite set of grid points. Discretization, however, incurs errors and leads to deteriorated recovery performance. To address this issue, we propose an iterative reweighted method which jointly estimates the unknown parameters and the sparse signals. Specifically, the proposed algorithm is developed by iteratively decreasing a surrogate function majorizing a given objective function, which results in a gradual and interweaved iterative process to refine the unknown parameters and the sparse signal. Numerical results show that the algorithm provides superior performance in resolving closely-spaced frequency components

Pattern Coupled Sparse Bayesian Learning for Recovery of Time Varying Sparse Signals

We consider the problem of recovering block-sparse signals whose structures are unknown \emph{a priori}. Block-sparse signals with nonzero coefficients occurring in clusters arise naturally in many practical scenarios. However, the knowledge of the block structure is usually unavailable in practice. In this paper, we develop a new sparse Bayesian learning method for recovery of block-sparse signals with unknown cluster patterns. Specifically, a pattern-coupled hierarchical Gaussian prior model is introduced to characterize the statistical dependencies among coefficients, in which a set of hyperparameters are employed to control the sparsity of signal coefficients. Unlike the conventional sparse Bayesian learning framework in which each individual hyperparameter is associated independently with each coefficient, in this paper, the prior for each coefficient not only involves its own hyperparameter, but also the hyperparameters of its immediate neighbors. In doing this way, the sparsity patterns of neighboring coefficients are related to each other and the hierarchical model has the potential to encourage structured-sparse solutions. The hyperparameters, along with the sparse signal, are learned by maximizing their posterior probability via an expectation-maximization (EM) algorithm. Numerical results show that the proposed algorithm presents uniform superiority over other existing methods in a series of experiments

On the Convergence of Memory-Based Distributed SGD

Distributed stochastic gradient descent~(DSGD) has been widely used for optimizing large-scale machine learning models, including both convex and non-convex models. With the rapid growth of model size, huge communication cost has been the bottleneck of traditional DSGD. Recently, many communication compression methods have been proposed. Memory-based distributed stochastic gradient descent~(M-DSGD) is one of the efficient methods since each worker communicates a sparse vector in each iteration so that the communication cost is small. Recent works propose the convergence rate of M-DSGD when it adopts vanilla SGD. However, there is still a lack of convergence theory for M-DSGD when it adopts momentum SGD. In this paper, we propose a universal convergence analysis for M-DSGD by introducing \emph{transformation equation}. The transformation equation describes the relation between traditional DSGD and M-DSGD so that we can transform M-DSGD to its corresponding DSGD. Hence we get the convergence rate of M-DSGD with momentum for both convex and non-convex problems. Furthermore, we combine M-DSGD and stagewise learning that the learning rate of M-DSGD in each stage is a constant and is decreased by stage, instead of iteration. Using the transformation equation, we propose the convergence rate of stagewise M-DSGD which bridges the gap between theory and practice

Generation of high-energy clean multicolored ultrashort pulses and their application in single-shot temporal contrast measurement

We demonstrate the generation of 100-{\mu}J-level multicolored femtosecond pulses based on a single-stage cascaded four-wave mixing (CFWM) process in a thin glass plate. The generated high-energy CFWM signals can shift the central wavelength and have well-enhanced temporal contrast because of the third-order nonlinear process. They are innovatively used as clean sampling pulses of a cross-correlator for single-shot temporal contrast measurement. With a simple home-made setup, the proof-of-principle experimental results demonstrate the single-shot cross-correlator with dynamic range of 1010, temporal resolution of about 160 fs and temporal window of 50 ps. To the best of our knowledge, this is the first demonstration in which both the dynamic range and the temporal resolution of a single-shot temporal contrast measurement are comparable to those of a commercial delay-scanning cross-correlator.Comment: 20 pages, 8 figure

Pattern-Coupled Sparse Bayesian Learning for Recovery of Block-Sparse Signals

We consider the problem of recovering block-sparse signals whose structures are unknown \emph{a priori}. Block-sparse signals with nonzero coefficients occurring in clusters arise naturally in many practical scenarios. However, the knowledge of the block structure is usually unavailable in practice. In this paper, we develop a new sparse Bayesian learning method for recovery of block-sparse signals with unknown cluster patterns. Specifically, a pattern-coupled hierarchical Gaussian prior model is introduced to characterize the statistical dependencies among coefficients, in which a set of hyperparameters are employed to control the sparsity of signal coefficients. Unlike the conventional sparse Bayesian learning framework in which each individual hyperparameter is associated independently with each coefficient, in this paper, the prior for each coefficient not only involves its own hyperparameter, but also the hyperparameters of its immediate neighbors. In doing this way, the sparsity patterns of neighboring coefficients are related to each other and the hierarchical model has the potential to encourage structured-sparse solutions. The hyperparameters, along with the sparse signal, are learned by maximizing their posterior probability via an expectation-maximization (EM) algorithm. Numerical results show that the proposed algorithm presents uniform superiority over other existing methods in a series of experiments

A New Class of Parametrization for Dark Energy without Divergence

In this paper, we propose a new class of parametrization of the equation of state of dark energy. In contrast with the famous CPL parametrization, these new parametrization of the equation of state does not divergent during the evolution of the Universe even in the future. Also, we perform a observational constraint on two simplest dark energy models belonging to this new class of parametrization, by using the Markov Chain Monte Carlo (MCMC) method and the combined latest observational data from the type Ia supernova compilations including Union2(557), cosmic microwave background, and baryon acoustic oscillation.Comment: 8 pages, 4 figure

Scalable Stochastic Alternating Direction Method of Multipliers

Stochastic alternating direction method of multipliers (ADMM), which visits only one sample or a mini-batch of samples each time, has recently been proved to achieve better performance than batch ADMM. However, most stochastic methods can only achieve a convergence rate $O(1/\sqrt T)$ on general convex problems,where T is the number of iterations. Hence, these methods are not scalable with respect to convergence rate (computation cost). There exists only one stochastic method, called SA-ADMM, which can achieve convergence rate $O(1/T)$ on general convex problems. However, an extra memory is needed for SA-ADMM to store the historic gradients on all samples, and thus it is not scalable with respect to storage cost. In this paper, we propose a novel method, called scalable stochastic ADMM(SCAS-ADMM), for large-scale optimization and learning problems. Without the need to store the historic gradients, SCAS-ADMM can achieve the same convergence rate $O(1/T)$ as the best stochastic method SA-ADMM and batch ADMM on general convex problems. Experiments on graph-guided fused lasso show that SCAS-ADMM can achieve state-of-the-art performance in real application

Controllable growth of centimeter-size 2D perovskite heterostructural single crystals for highly narrow dual-band photodetectors

Two-dimensional (2D) organic-inorganic perovskites have recently attracted increasing attention due to their great environmental stability, remarkable quantum confinement effect and layered characteristic. Heterostructures consisting of 2D layered perovskites are expected to exhibit new physical phenomena inaccessible to the single 2D perovskites and can greatly extend their functionalities for novel electronic and optoelectronic applications. Herein, we develop a novel solution method to synthesize 2D perovskite single-crystals with the centimeter size, high phase purity, controllable junction depth, high crystalline quality and great stability for highly narrow dual-band photodetectors. On the basis of the different lattice constant, solubility and growth rate between different n number, the newly designed synthesis method allows to first grow n=1 perovskite guided by the self-assembled layer of the organic cations at the water-air interface and subsequently n=2 layer is formed via diffusion process. Such growth process provides an efficient away for us to readily obtain 2D perovskite heterostructural single-crystals with various thickness and junction depth by controlling the concentration, reaction temperature and time. Photodetectors based on such heterostructural single crystal plates exhibit extremely low dark current, high on-off current ratio, and highly narrow dual-band spectral response with a full-width at half-maximum of 20 nm at 540 nm and 34 nm at 610 nm. In particular, the synthetic strategy is general for other 2D perovskites and the narrow dual-band spectral response with all full-width at half-maximum below 40 nm can be continuously tuned from red to blue by properly changing the halide compositions