Orthogonal AMP

Approximate message passing (AMP) is a low-cost iterative signal recovery algorithm for linear system models. When the system transform matrix has independent identically distributed (IID) Gaussian entries, the performance of AMP can be asymptotically characterized by a simple scalar recursion called state evolution (SE). However, SE may become unreliable for other matrix ensembles, especially for ill-conditioned ones. This imposes limits on the applications of AMP. In this paper, we propose an orthogonal AMP (OAMP) algorithm based on de-correlated linear estimation (LE) and divergence-free non-linear estimation (NLE). The Onsager term in standard AMP vanishes as a result of the divergence-free constraint on NLE. We develop an SE procedure for OAMP and show numerically that the SE for OAMP is accurate for general unitarily-invariant matrices, including IID Gaussian matrices and partial orthogonal matrices. We further derive optimized options for OAMP and show that the corresponding SE fixed point coincides with the optimal performance obtained via the replica method. Our numerical results demonstrate that OAMP can be advantageous over AMP, especially for ill-conditioned matricesComment: accepted for publication in IEEE Acces

Penalized Clustering of Large Scale Functional Data with Multiple Covariates

In this article, we propose a penalized clustering method for large scale data with multiple covariates through a functional data approach. In the proposed method, responses and covariates are linked together through nonparametric multivariate functions (fixed effects), which have great flexibility in modeling a variety of function features, such as jump points, branching, and periodicity. Functional ANOVA is employed to further decompose multivariate functions in a reproducing kernel Hilbert space and provide associated notions of main effect and interaction. Parsimonious random effects are used to capture various correlation structures. The mixed-effect models are nested under a general mixture model, in which the heterogeneity of functional data is characterized. We propose a penalized Henderson's likelihood approach for model-fitting and design a rejection-controlled EM algorithm for the estimation. Our method selects smoothing parameters through generalized cross-validation. Furthermore, the Bayesian confidence intervals are used to measure the clustering uncertainty. Simulation studies and real-data examples are presented to investigate the empirical performance of the proposed method. Open-source code is available in the R package MFDA

An ensemble of random graphs with identical degree distribution

Degree distribution, or equivalently called degree sequence, has been commonly used to be one of most significant measures for studying a large number of complex networks with which some well-known results have been obtained. By contrast, in this paper, we report a fact that two arbitrarily chosen networks with identical degree distribution can have completely different other topological structure, such as diameter, spanning trees number, pearson correlation coefficient, and so forth. Besides that, for a given degree distribution (as power-law distribution with exponent $\gamma=3$ discussed here), it is reasonable to ask how many network models with such a constraint we can have. To this end, we generate an ensemble of this kind of random graphs with $P(k)\sim k^{-\gamma}$ ($\gamma=3$), denoted as graph space $\mathcal{N}(p,q,t)$ where probability parameters $p$ and $q$ hold on $p+q=1$, and indirectly show the cardinality of $\mathcal{N}(p,q,t)$ seems to be large enough in the thermodynamics limit, i.e., $N\rightarrow\infty$, by varying values of $p$ and $q$. From the theoretical point of view, given an ultrasmall constant $p_{c}$, perhaps only graph model $N(1,0,t)$ is small-world and other are not in terms of diameter. And then, we study spanning trees number on two deterministic graph models and obtain both upper bound and lower bound for other members. Meanwhile, for arbitrary $p(\neq1)$, we prove that graph model $N(p,q,t)$ does go through two phase transitions over time, i.e., starting by non-assortative pattern and then suddenly going into disassortative region, and gradually converging to initial place (non-assortative point). Among of them, one "null" graph model is built

On the Performance of Turbo Signal Recovery with Partial DFT Sensing Matrices

This letter is on the performance of the turbo signal recovery (TSR) algorithm for partial discrete Fourier transform (DFT) matrices based compressed sensing. Based on state evolution analysis, we prove that TSR with a partial DFT sensing matrix outperforms the well-known approximate message passing (AMP) algorithm with an independent identically distributed (IID) sensing matrix.Comment: to appear in IEEE Signal Processing Letter

Strong transmission and reflection of edge modes in bounded photonic graphene

The propagation of linear and nonlinear edge modes in bounded photonic honeycomb lattices formed by an array of rapidly varying helical waveguides is studied. These edge modes are found to exhibit strong transmission (reflection) around sharp corners when the dispersion relation is topologically nontrivial (trivial), and can also remain stationary. An asymptotic theory is developed that establishes the presence (absence) of edge states on all four sides, including in particular armchair edge states, in the topologically nontrivial (trivial) case. In the presence of topological protection, nonlinear edge solitons can persist over very long distances.Comment: 5 pages, 4 figures. Minor updates on the presentation and interpretation of results. The movies showing transmission and reflection of linear edge modes are available at https://www.youtube.com/watch?v=XhaZZlkMadQ and https://www.youtube.com/watch?v=R8NOw0NvRu

A Statistical Perspective on Algorithmic Leveraging

One popular method for dealing with large-scale data sets is sampling. For example, by using the empirical statistical leverage scores as an importance sampling distribution, the method of algorithmic leveraging samples and rescales rows/columns of data matrices to reduce the data size before performing computations on the subproblem. This method has been successful in improving computational efficiency of algorithms for matrix problems such as least-squares approximation, least absolute deviations approximation, and low-rank matrix approximation. Existing work has focused on algorithmic issues such as worst-case running times and numerical issues associated with providing high-quality implementations, but none of it addresses statistical aspects of this method. In this paper, we provide a simple yet effective framework to evaluate the statistical properties of algorithmic leveraging in the context of estimating parameters in a linear regression model with a fixed number of predictors. We show that from the statistical perspective of bias and variance, neither leverage-based sampling nor uniform sampling dominates the other. This result is particularly striking, given the well-known result that, from the algorithmic perspective of worst-case analysis, leverage-based sampling provides uniformly superior worst-case algorithmic results, when compared with uniform sampling. Based on these theoretical results, we propose and analyze two new leveraging algorithms. A detailed empirical evaluation of existing leverage-based methods as well as these two new methods is carried out on both synthetic and real data sets. The empirical results indicate that our theory is a good predictor of practical performance of existing and new leverage-based algorithms and that the new algorithms achieve improved performance.Comment: 44 pages, 17 figure