Learning without the Phase: Regularized PhaseMax Achieves Optimal Sample Complexity

The problem of estimating an unknown signal, x_0 ϵ R^n, from a vector y ϵ R^m consisting of m magnitude-only measurements of the form y_i = |a_ix_o|, where a_i’s are the rows of a known measurement matrix A is a classical problem known as phase retrieval. This problem arises when measuring the phase is costly or altogether infeasible. In many applications in machine learning, signal processing, statistics, etc., the underlying signal has certain structure (sparse, low-rank, finite alphabet, etc.), opening of up the possibility of recovering x_0 from a number of measurements smaller than the ambient dimension, i.e., m < n. Ideally, one would like to recover the signal from a number of phaseless measurements that is on the order of the "degrees of freedom" of the structured x_0. To this end, inspired by the PhaseMax algorithm, we formulate a convex optimization problem, where the objective function relies on an initial estimate of the true signal and also includes an additive regularization term to encourage structure. The new formulation is referred to as regularized PhaseMax. We analyze the performance of regularized PhaseMax to find the minimum number of phaseless measurements required for perfect signal recovery. The results are asymptotic and are in terms of the geometrical properties (such as the Gaussian width) of certain convex cones. When the measurement matrix has i.i.d. Gaussian entries, we show that our proposed method is indeed order-wise optimal, allowing perfect recovery from a number of phaseless measurements that is only a constant factor away from the degrees of freedom. We explicitly compute this constant factor, in terms of the quality of the initial estimate, by deriving the exact phase transition. The theory well matches empirical results from numerical simulations

Learning without the Phase: Regularized PhaseMax Achieves Optimal Sample Complexity

The problem of estimating an unknown signal, x_0 ϵ R^n, from a vector y ϵ R^m consisting of m magnitude-only measurements of the form y_i = |a_ix_o|, where a_i’s are the rows of a known measurement matrix A is a classical problem known as phase retrieval. This problem arises when measuring the phase is costly or altogether infeasible. In many applications in machine learning, signal processing, statistics, etc., the underlying signal has certain structure (sparse, low-rank, finite alphabet, etc.), opening of up the possibility of recovering x_0 from a number of measurements smaller than the ambient dimension, i.e., m < n. Ideally, one would like to recover the signal from a number of phaseless measurements that is on the order of the "degrees of freedom" of the structured x_0. To this end, inspired by the PhaseMax algorithm, we formulate a convex optimization problem, where the objective function relies on an initial estimate of the true signal and also includes an additive regularization term to encourage structure. The new formulation is referred to as regularized PhaseMax. We analyze the performance of regularized PhaseMax to find the minimum number of phaseless measurements required for perfect signal recovery. The results are asymptotic and are in terms of the geometrical properties (such as the Gaussian width) of certain convex cones. When the measurement matrix has i.i.d. Gaussian entries, we show that our proposed method is indeed order-wise optimal, allowing perfect recovery from a number of phaseless measurements that is only a constant factor away from the degrees of freedom. We explicitly compute this constant factor, in terms of the quality of the initial estimate, by deriving the exact phase transition. The theory well matches empirical results from numerical simulations

Structured Signal Recovery from Nonlinear Measurements with Applications in Phase Retrieval and Linear Classification

Nonlinear models are widely used in signal processing, statistics, and machine learning to model real-world applications. A popular class of such models is the single-index model where the response variable is related to a linear combination of dependent variables through a link function. In other words, if x ∈ Rp denotes the input signal, the posterior mean of the generated output y has the form, E[y|x] = ρ(xTw), where ρ :R → R is a known function (referred to as the link function), and w ∈ Rp is the vector of unknown parameters. When ρ(•) is invertible, this class of models is called generalized linear models (GLMs). GLMs are commonly used in statistics and are often viewed as flexible generalizations of linear regression. Given n measurements (samples) from this model, D = {(xi, yi) | 1 ≤q i ≤ n}, the goal is to estimate the parameter vector w. While the model parameters are assumed to be unknown, in many applications these parameters follow certain structures (sparse, low-rank, group-sparse, etc.) The knowledge on this structure can be used to form more accurate estimators. The main contribution of this thesis is to provide a precise performance analysis for convex optimization programs that are used for parameter estimation in two important classes of single-index models. These classes are: (1) phase retrieval in signal processing, and (2) binary classification in statistical learning. The first class of models studied in this thesis is the phase retrieval problem, where the goal is to recover a discrete complex-valued signal from amplitudes of its linear combinations. Methods based on convex optimization have recently gained significant attentions in the literature. The conventional convex-optimization-based methods resort to the idea of lifting which makes them computationally inefficient. In addition to providing an analysis of the recovery threshold for the semidefinite-programming-based methods, this thesis studies the performance of a new convex relaxation for the phase retrieval problem, known as phasemax, which is computationally more efficient as it does not lift the signal to higher dimensions. Furthermore, to address the case of structured signals, regularized phasemax is introduced along with a precise characterization of the conditions for its perfect recovery in the asymptotic regime. The next important application studied in this thesis is the binary classification in statistical learning. While classification models have been studied in the literature since 1950's, the understanding of their performance has been incomplete until very recently. Inspired by the maximum likelihood (ML) estimator in logistic models, we analyze a class of optimization programs that attempts to find the model parameters by minimizing an objective that consists of a loss function (which is often inspired by the ML estimator) and an additive regularization term that enforces our knowledge on the structure. There are two operating regimes for this problem depending on the separability of the training data set D. In the asymptotic regime, where the number of samples and the number of parameters grow to infinity, a phase transition phenomenon is demonstrated that happens at a certain over-parameterization ratio. We compute this phase transition for the setting where the underlying data is drawn from a Gaussian distribution. In the case where the data is non-separable, the ML estimator is well-defined, and its attributes have been studied in the classical statistics. However, these classical results fail to provide reasonable estimate in the regime where the number of data points is proportional to the number of samples. One contribution of this thesis is to provide an exact analysis on the performance of the regularized logistic regression when the number of training data is proportional to the number of samples. When the data is separable (a.k.a. the interpolating regime), there exist multiple linear classifiers that perfectly fit the training data. In this regime, we introduce and analyze the performance of "extended margin maximizers" (EMMs). Inspired by the max-margin classifier, EMM classifiers simultaneously consider maximizing the margin and the structure of the parameter. Lastly, we discuss another generalization to the max-margin classifier, referred to as the robust max-margin classifier, that takes into account the perturbations by an adversary. It is shown that for a broad class of loss functions, gradient descent iterates (with proper step sizes) converge to the robust max-margin classifier.</p

Development of Hybrid Sol-Gel Coatings on AA2024-T3 with Environmentally Benign Corrosion Inhibitors

Aluminium alloys are still considered as one of the primary light alloys with high strength that can be used in aerospace and marine structure with a moderate economic cost. However, aluminium alloys are affected by atmospheric and marine corrosion which reduces their overall reliability. The application of coatings is one of the strategies for mitigating the corrosion on aluminium alloys. Sol-gel technology is one of the coating strategies with potential for excellent chemical and environment stability, providing eco-friendly performance improvement. Sol-gel coatings can enhance corrosion resistance by adopting many corrosion inhibitors that can protect aluminium alloys substrates from corrosion. In this project, three approaches are developed and studied to protect AA 2024 in high salinity of 3.5% NaCl environment. The first approach involved studying the direct application of film-forming environmentally benign corrosion inhibitors on the surface of the substrate without any involvement of the coating, these inhibitors are benzimidazole (BZI) and oleic acid (OA), and were studied alone and in combination. The second approach, investigated the corrosion protection of two sets of novel hybrid organic-inorganic sol-gel derived coatings using low-temperature cure processes at 80°C. The base sol-gel formula was made using alkoxide silane-based precursors including; tetraethylorthosilicate silane (TEOS) and trimethoxymethyl silane (MTMS) and this was labelled as (SBX- 80). The other enhanced formula was developed by adding the fluorinated precursor, 1H.1H.2H.2Hperfluorodecyltriethoxy silane (PFOTS), to the base SBX-80 formula to synthesis the fluorinated-sol-gel formula and this was labelled as (F-SBX-80). The third approach investigated the addition of corrosion inhibitors of benzimidazole (BZI) and oleic acid (OA) to the base SBX formula to increase the corrosion protection, either alone or in combination. All sol-gel derived coating systems, including those modified with BZI or OA corrosion inhibitors exhibited excellent corrosion protection as evidenced visually, or by electrochemical methods with various levels of stability when immersed in saline solution. The coatings exhibited high capacitance and resistance that might provide active/barrier corrosion protection at small thicknesses less than 20 μm. Furthermore, there was an enhancement on mechanical properties of coating films with insignificant effect on the adhesion to the aluminium alloy substrate combined with excellent cracking-resistance. There was also a significant increase in surface water contact angle for these systems, which indicates the potential for enhanced easy-cleaning features