Stochastic Optimization of PCA with Capped MSG

We study PCA as a stochastic optimization problem and propose a novel stochastic approximation algorithm which we refer to as "Matrix Stochastic Gradient" (MSG), as well as a practical variant, Capped MSG. We study the method both theoretically and empirically

Permutation and Grouping Methods for Sharpening Gaussian Process Approximations

Vecchia's approximate likelihood for Gaussian process parameters depends on how the observations are ordered, which can be viewed as a deficiency because the exact likelihood is permutation-invariant. This article takes the alternative standpoint that the ordering of the observations can be tuned to sharpen the approximations. Advantageously chosen orderings can drastically improve the approximations, and in fact, completely random orderings often produce far more accurate approximations than default coordinate-based orderings do. In addition to the permutation results, automatic methods for grouping calculations of components of the approximation are introduced, having the result of simultaneously improving the quality of the approximation and reducing its computational burden. In common settings, reordering combined with grouping reduces Kullback-Leibler divergence from the target model by a factor of 80 and computation time by a factor of 2 compared to ungrouped approximations with default ordering. The claims are supported by theory and numerical results with comparisons to other approximations, including tapered covariances and stochastic partial differential equation approximations. Computational details are provided, including efficiently finding the orderings and ordered nearest neighbors, and profiling out linear mean parameters and using the approximations for prediction and conditional simulation. An application to space-time satellite data is presented

On Longest Repeat Queries Using GPU

Repeat finding in strings has important applications in subfields such as computational biology. The challenge of finding the longest repeats covering particular string positions was recently proposed and solved by \.{I}leri et al., using a total of the optimal $O(n)$ time and space, where $n$ is the string size. However, their solution can only find the \emph{leftmost} longest repeat for each of the $n$ string position. It is also not known how to parallelize their solution. In this paper, we propose a new solution for longest repeat finding, which although is theoretically suboptimal in time but is conceptually simpler and works faster and uses less memory space in practice than the optimal solution. Further, our solution can find \emph{all} longest repeats of every string position, while still maintaining a faster processing speed and less memory space usage. Moreover, our solution is \emph{parallelizable} in the shared memory architecture (SMA), enabling it to take advantage of the modern multi-processor computing platforms such as the general-purpose graphics processing units (GPU). We have implemented both the sequential and parallel versions of our solution. Experiments with both biological and non-biological data show that our sequential and parallel solutions are faster than the optimal solution by a factor of 2--3.5 and 6--14, respectively, and use less memory space.Comment: 14 page

Resource Allocation for Delay Differentiated Traffic in Multiuser OFDM Systems

Most existing work on adaptive allocation of subcarriers and power in multiuser orthogonal frequency division multiplexing (OFDM) systems has focused on homogeneous traffic consisting solely of either delay-constrained data (guaranteed service) or non-delay-constrained data (best-effort service). In this paper, we investigate the resource allocation problem in a heterogeneous multiuser OFDM system with both delay-constrained (DC) and non-delay-constrained (NDC) traffic. The objective is to maximize the sum-rate of all the users with NDC traffic while maintaining guaranteed rates for the users with DC traffic under a total transmit power constraint. Through our analysis we show that the optimal power allocation over subcarriers follows a multi-level water-filling principle; moreover, the valid candidates competing for each subcarrier include only one NDC user but all DC users. By converting this combinatorial problem with exponential complexity into a convex problem or showing that it can be solved in the dual domain, efficient iterative algorithms are proposed to find the optimal solutions. To further reduce the computational cost, a low-complexity suboptimal algorithm is also developed. Numerical studies are conducted to evaluate the performance the proposed algorithms in terms of service outage probability, achievable transmission rate pairs for DC and NDC traffic, and multiuser diversity.Comment: 29 pages, 8 figures, submitted to IEEE Transactions on Wireless Communication