Penalized Orthogonal Iteration for Sparse Estimation of Generalized Eigenvalue Problem

We propose a new algorithm for sparse estimation of eigenvectors in generalized eigenvalue problems (GEP). The GEP arises in a number of modern data-analytic situations and statistical methods, including principal component analysis (PCA), multiclass linear discriminant analysis (LDA), canonical correlation analysis (CCA), sufficient dimension reduction (SDR) and invariant co-ordinate selection. We propose to modify the standard generalized orthogonal iteration with a sparsity-inducing penalty for the eigenvectors. To achieve this goal, we generalize the equation-solving step of orthogonal iteration to a penalized convex optimization problem. The resulting algorithm, called penalized orthogonal iteration, provides accurate estimation of the true eigenspace, when it is sparse. Also proposed is a computationally more efficient alternative, which works well for PCA and LDA problems. Numerical studies reveal that the proposed algorithms are competitive, and that our tuning procedure works well. We demonstrate applications of the proposed algorithm to obtain sparse estimates for PCA, multiclass LDA, CCA and SDR. Supplementary materials are available online

Kernel methods for detecting coherent structures in dynamical data

We illustrate relationships between classical kernel-based dimensionality reduction techniques and eigendecompositions of empirical estimates of reproducing kernel Hilbert space (RKHS) operators associated with dynamical systems. In particular, we show that kernel canonical correlation analysis (CCA) can be interpreted in terms of kernel transfer operators and that it can be obtained by optimizing the variational approach for Markov processes (VAMP) score. As a result, we show that coherent sets of particle trajectories can be computed by kernel CCA. We demonstrate the efficiency of this approach with several examples, namely the well-known Bickley jet, ocean drifter data, and a molecular dynamics problem with a time-dependent potential. Finally, we propose a straightforward generalization of dynamic mode decomposition (DMD) called coherent mode decomposition (CMD). Our results provide a generic machine learning approach to the computation of coherent sets with an objective score that can be used for cross-validation and the comparison of different methods

Optimal wavy surface to suppress vortex shedding using second-order sensitivity to shape changes

A method to find optimal 2nd-order perturbations is presented, and applied to find the optimal spanwise-wavy surface for suppression of cylinder wake instability. Second-order perturbations are required to capture the stabilizing effect of spanwise waviness, which is ignored by standard adjoint-based sensitivity analyses. Here, previous methods are extended so that (i) 2nd-order sensitivity is formulated for base flow changes satisfying linearised Navier-Stokes, and (ii) the resulting method is applicable to a 2D global instability problem. This makes it possible to formulate 2nd-order sensitivity to shape modifications. Using this formulation, we find the optimal shape to suppress the a cylinder wake instability. The optimal shape is then perturbed by random distributions in full 3D stability analysis to confirm that it is a local optimal at the given amplitude and wavelength. Furthermore, it is shown that none of the 10 random wavy shapes alone stabilize the wake flow at Re=50, while the optimal shape does. At Re=100, surface waviness of maximum height 1% of the cylinder diameter is sufficient to stabilize the flow. The optimal surface creates streaks by passively extracting energy from the base flow derivatives and effectively altering the tangential velocity component at the wall, as opposed to spanwise-wavy suction which inputs energy to the normal velocity component at the wall. This paper presents a fully two-dimensional and computationally affordable method to find optimal 2nd-order perturbations of generic flow instability problems and any boundary control (such as boundary forcing, shape modulation or suction).Comment: 19 pages, 6 figure