Infeasible Deterministic, Stochastic, and Variance-Reduction Algorithms for Optimization under Orthogonality Constraints

Orthogonality constraints naturally appear in many machine learning problems, from Principal Components Analysis to robust neural network training. They are usually solved using Riemannian optimization algorithms, which minimize the objective function while enforcing the constraint. However, enforcing the orthogonality constraint can be the most time-consuming operation in such algorithms. Recently, Ablin & Peyr\'e (2022) proposed the Landing algorithm, a method with cheap iterations that does not enforce the orthogonality constraint but is attracted towards the manifold in a smooth manner. In this article, we provide new practical and theoretical developments for the landing algorithm. First, the method is extended to the Stiefel manifold, the set of rectangular orthogonal matrices. We also consider stochastic and variance reduction algorithms when the cost function is an average of many functions. We demonstrate that all these methods have the same rate of convergence as their Riemannian counterparts that exactly enforce the constraint. Finally, our experiments demonstrate the promise of our approach to an array of machine-learning problems that involve orthogonality constraints

A Riemannian Proximal Newton Method

In recent years, the proximal gradient method and its variants have been generalized to Riemannian manifolds for solving optimization problems with an additively separable structure, i.e., $f + h$, where $f$ is continuously differentiable, and $h$ may be nonsmooth but convex with computationally reasonable proximal mapping. In this paper, we generalize the proximal Newton method to embedded submanifolds for solving the type of problem with $h(x) = \mu \|x\|_1$. The generalization relies on the Weingarten and semismooth analysis. It is shown that the Riemannian proximal Newton method has a local superlinear convergence rate under certain reasonable assumptions. Moreover, a hybrid version is given by concatenating a Riemannian proximal gradient method and the Riemannian proximal Newton method. It is shown that if the objective function satisfies the Riemannian KL property and the switch parameter is chosen appropriately, then the hybrid method converges globally and also has a local superlinear convergence rate. Numerical experiments on random and synthetic data are used to demonstrate the performance of the proposed methods

Direct Exoplanet Detection Using L1 Norm Low-Rank Approximation

We propose to use low-rank matrix approximation using the component-wise L1-norm for direct imaging of exoplanets. Exoplanet detection by direct imaging is a challenging task for three main reasons: (1) the host star is several orders of magnitude brighter than exoplanets, (2) the angular distance between exoplanets and star is usually very small, and (3) the images are affected by the noises called speckles that are very similar to the exoplanet signal both in shape and intensity. We first empirically examine the statistical noise assumptions of the L1 and L2 models, and then we evaluate the performance of the proposed L1 low-rank approximation (L1-LRA) algorithm based on visual comparisons and receiver operating characteristic (ROC) curves. We compare the results of the L1-LRA with the widely used truncated singular value decomposition (SVD) based on the L2 norm in two different annuli, one close to the star and one far away.Comment: 13 pages, 4 figures, BNAIC/BeNeLearn 202

Multispectral snapshot demosaicing via non-convex matrix completion

Snapshot mosaic multispectral imagery acquires an undersampled data cube by acquiring a single spectral measurement per spatial pixel. Sensors which acquire $p$ frequencies, therefore, suffer from severe $1/p$ undersampling of the full data cube. We show that the missing entries can be accurately imputed using non-convex techniques from sparse approximation and matrix completion initialised with traditional demosaicing algorithms. In particular, we observe the peak signal-to-noise ratio can typically be improved by 2 to 5 dB over current state-of-the-art methods when simulating a $p=16$ mosaic sensor measuring both high and low altitude urban and rural scenes as well as ground-based scenes.Comment: 5 pages, 2 figures, 1 tabl

The (1+1)-dimensional Massive sine-Gordon Field Theory and the Gaussian Wave-functional Approach

The ground, one- and two-particle states of the (1+1)-dimensional massive sine-Gordon field theory are investigated within the framework of the Gaussian wave-functional approach. We demonstrate that for a certain region of the model-parameter space, the vacuum of the field system is asymmetrical. Furthermore, it is shown that two-particle bound state can exist upon the asymmetric vacuum for a part of the aforementioned region. Besides, for the bosonic equivalent to the massive Schwinger model, the masses of the one boson and two-boson bound states agree with the recent second-order results of a fermion-mass perturbation calculation when the fermion mass is small.Comment: Latex, 11 pages, 8 figures (EPS files

Mitochondrial dysfunction and biogenesis: do ICU patients die from mitochondrial failure?

Mitochondrial functions include production of energy, activation of programmed cell death, and a number of cell specific tasks, e.g., cell signaling, control of Ca2+ metabolism, and synthesis of a number of important biomolecules. As proper mitochondrial function is critical for normal performance and survival of cells, mitochondrial dysfunction often leads to pathological conditions resulting in various human diseases. Recently mitochondrial dysfunction has been linked to multiple organ failure (MOF) often leading to the death of critical care patients. However, there are two main reasons why this insight did not generate an adequate resonance in clinical settings. First, most data regarding mitochondrial dysfunction in organs susceptible to failure in critical care diseases (liver, kidney, heart, lung, intestine, brain) were collected using animal models. Second, there is no clear therapeutic strategy how acquired mitochondrial dysfunction can be improved. Only the benefit of such therapies will confirm the critical role of mitochondrial dysfunction in clinical settings. Here we summarized data on mitochondrial dysfunction obtained in diverse experimental systems, which are related to conditions seen in intensive care unit (ICU) patients. Particular attention is given to mechanisms that cause cell death and organ dysfunction and to prospective therapeutic strategies, directed to recover mitochondrial function. Collectively the data discussed in this review suggest that appropriate diagnosis and specific treatment of mitochondrial dysfunction in ICU patients may significantly improve the clinical outcome

An Alternating Minimization Algorithm with Trajectory for Direct Exoplanet Detection

Effective image post-processing algorithms are vital for the successful direct imaging of exoplanets. Existing algorithms use techniques based on a low-rank approximation to separate the rotating planet signal from the quasi-static speckles. In this paper, we present a novel approach that iteratively finds the planet’s flux and the low-rank approximation of quasi-static signals, strengthening the existing model based on lowrank approximations. We implement the algorithm with two different norms and test it on data, showing improvement over classical low-rank approaches. Our results highlight the benefits of iterative refinement of low-rank approximation to enhance planet detection