Non-convex Optimization for Machine Learning

A vast majority of machine learning algorithms train their models and perform inference by solving optimization problems. In order to capture the learning and prediction problems accurately, structural constraints such as sparsity or low rank are frequently imposed or else the objective itself is designed to be a non-convex function. This is especially true of algorithms that operate in high-dimensional spaces or that train non-linear models such as tensor models and deep networks. The freedom to express the learning problem as a non-convex optimization problem gives immense modeling power to the algorithm designer, but often such problems are NP-hard to solve. A popular workaround to this has been to relax non-convex problems to convex ones and use traditional methods to solve the (convex) relaxed optimization problems. However this approach may be lossy and nevertheless presents significant challenges for large scale optimization. On the other hand, direct approaches to non-convex optimization have met with resounding success in several domains and remain the methods of choice for the practitioner, as they frequently outperform relaxation-based techniques - popular heuristics include projected gradient descent and alternating minimization. However, these are often poorly understood in terms of their convergence and other properties. This monograph presents a selection of recent advances that bridge a long-standing gap in our understanding of these heuristics. The monograph will lead the reader through several widely used non-convex optimization techniques, as well as applications thereof. The goal of this monograph is to both, introduce the rich literature in this area, as well as equip the reader with the tools and techniques needed to analyze these simple procedures for non-convex problems.Comment: The official publication is available from now publishers via http://dx.doi.org/10.1561/220000005

Metal organic frameworks as heterogenous nitric oxide catalysts for use in the development of therapeutic polymer materials

2014 Fall.Includes bibliographical references.Implantable polymeric medical devices are subject to surface biofouling due to the deposition of microbial agents and the accumulation of proteins at the material interface. Consequently, medical devices which are intended for beneficial functions can become a potentially fatal threat. As a result biofouling resistant materials are vigorously sought through the manipulation of material surface properties and by eluting therapeutics on the material surface. Nitric oxide (NO) is a bioactive agent generated by most nucleated cells in the human body and is known to mediate antimicrobial and antithrombus effects while maintain the capacity to promote the proliferation of healthy tissues. As such, the development of NO releasing biomaterials is known to reduce incidences of surface biofouling. However, current NO releasing materials are limited to short lifetimes of used based on limited capacity of exogenous NO which can be incorporated into the material. In order to circumvent this problem the goal of this research is to develop a biomaterial which generates NO from an endogenously supplied source. Metal organic frameworks (MOFs) were selected for investigation as heterogeneous catalysts for the generation of NO from bioavailable NO donors, S-nitrosothiols (RSNOS). MOFs were evaluated as NO catalysts based on their capacity to react with various RSNO substrates and their maintained structural integrity under reaction conditions. Presented herein is the successful demonstration of a Cu-MOF for the catalytic generation of NO from bioavailable RSNOs donors. However, the limited stability of this proof of principle MOF in aqueous solution prompted the development of a MOF-NO catalyst that is suitable for physiological applications through tuning the organic ligands used in the construction of the framework. Finally a two-fold demonstration of the feasibility towards designing composite MOF based biomaterials is presented as blended materials prepared via commercial manufacturing processes and via surface growth of MOFs on flexible polymeric substrates

Ice-Structure Interaction Analysis: Inverse Ice Force Prediction for Stiffened Plate and Dynamic Simulation

Offshore regions of the Arctic and the Great Lakes hold valuable resources in many respects for harvesting energy and serving as important shipping lanes. Ice loading poses a threat to structures in these regions with high local pressure and various failure modes. It is thus essential to evaluate the ice peak loadings using limited and site-specific data. This thesis aims to better predict the peak ice loading by developing an efficient inverse ice loading prediction methodology and accurate stiffened plate analysis for marine structure design. Additionally, the behavior of the ice-structure interaction is studied mathematically to understand the cyclic dynamic ice-loading applied on offshore structures during continuous ice crushing. Multiple inverse algorithms are presented for calculating the variable ice pressure acting on a stiffened steel plate. The analytical models are formulated to calculate the quasi-static pressure caused by contact of lake ice driven primarily by thermal expansion and winds. Loading pressures are calculated using strain measurements from a stiffened plate installed on a Keweenaw Peninsula lighthouse in Lake Superior. The ice sheet was essentially stationary through the winter months. The linear relationships between pressure and strain values are obtained by both strip beam theory and orthotropic plate theory. The inverse solutions are by nature not necessarily unique. Two inverse approaches using orthotropic plate theory show results with satisfying accuracy and efficiency compared to the finite element analysis. In addition, laboratory calibration and an examination using the recorded data from field measurements exhibit the effectiveness of the presented approach. Continuous ice brittle crushing occurs in the movement of an ice sheet against an offshore structure. Matlock’s ice-structure interaction model is used to simulate the behavior of the ice crushing by modeling ice teeth indentation contacting a spring-mass-dashpot structure. The dynamic behavior of the model is studied using Fourier analysis to predict the response of specific periodicity. The time histories of tooth deflections are expressed through non-linear dynamic equations. The kinematic initial conditions can be predicted at targeted periodicity via the Fourier analysis. Given a representative offshore wind tower system, the first mode shape of the physical system is calculated as input for the ice-structure interaction model as an extended validation. The amplitudes of the structural dynamic vibrations predicted by the analytical model at specific periodicity are compared to the mathematical numerical simulations. A discrete energy method is applied to accurately calculate the deformation of either unidirectional or cross-gridded stiffened panels. This approach obtains the strain energy of the plate and stiffeners using double Fourier series for the displacement fields. Two models are described assuming different reference planes. The first model presumes that the reference planes are located at the effective centroids which are calculated from the cross-sectional properties. The second model formulates the in-plane displacement fields at the mid-plane of the plate. The plate is simply supported along all four edges at the effective centroids for the first model, and at the mid-plane of the plate for the second model. Both methods accurately capture the deformations between stiffeners and the second model eliminates the complicated calculation for effective breadth which is an unavoidable effort for stiffened plate analysis using conventional orthotropic plate theory. The methods presented provide efficient design tools and can be applied to light weight structural design in various fields.PHDNaval Architecture & Marine EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/146006/1/yuxiz_1.pd

Compressed Sensing For Functional Magnetic Resonance Imaging Data

This thesis addresses the possibility of applying the compressed sensing (CS) framework to Functional Magnetic Resonance Imaging (fMRI) acquisition. The fMRI is one of the non-invasive neuroimaging technique that allows the brain activity to be captured and analysed in a living body. One disadvantage of fMRI is the trade-off between the spatial and temporal resolution of the data. To keep the experiments within a reasonable length of time, the current acquisition technique sacrifices the spatial resolution in favour of the temporal resolution. It is possible to improve this trade-off using compressed sensing. The main contribution of this thesis is to propose a novel reconstruction method, named Referenced Compressed Sensing, which exploits the redundancy between a signal and a correlated reference by using their distance as an objective function. The compressed video sequences reconstructed using Referenced CS have at least 50% higher in terms of Peak Signal-to-Noise Ratio (PSNR) compared to state-of-the-art conventional reconstruction methods. This thesis also addresses two issues related to Referenced CS. Firstly, the relationship between the reference and the reconstruction performance is studied. To maintain the high-quality references, the Running Gaussian Average (RGA) reference estimator is proposed. The reconstructed results have at least 3dB better PSNR performance with the use of RGA references. Secondly, the Referenced CS with Least Squares is proposed. This study shows that by incorporating the correlated reference, it is possible to perform a linear reconstruction as opposed to the iterative reconstruction commonly used in CS. This approach gives at least 19% improvement in PSNR compared to the state of the art, while reduces the computation time by at most 1200 times. The proposed method is applied to the fMRI data. This study shows that, using the same amount of samples, the data reconstructed using Referenced CS has higher resolution than the conventional acquisition technique and has on average 50% higher PSNR than state-of-the-art reconstructions. Lastly, to enhance the feature of interest in the fMRI data, the baseline independent (BI) analysis is proposed. Using the BI analysis shows up to 25% improvement in the accuracy of the Referenced CS feature

Control of quantum phenomena: Past, present, and future

Quantum control is concerned with active manipulation of physical and chemical processes on the atomic and molecular scale. This work presents a perspective of progress in the field of control over quantum phenomena, tracing the evolution of theoretical concepts and experimental methods from early developments to the most recent advances. The current experimental successes would be impossible without the development of intense femtosecond laser sources and pulse shapers. The two most critical theoretical insights were (1) realizing that ultrafast atomic and molecular dynamics can be controlled via manipulation of quantum interferences and (2) understanding that optimally shaped ultrafast laser pulses are the most effective means for producing the desired quantum interference patterns in the controlled system. Finally, these theoretical and experimental advances were brought together by the crucial concept of adaptive feedback control, which is a laboratory procedure employing measurement-driven, closed-loop optimization to identify the best shapes of femtosecond laser control pulses for steering quantum dynamics towards the desired objective. Optimization in adaptive feedback control experiments is guided by a learning algorithm, with stochastic methods proving to be especially effective. Adaptive feedback control of quantum phenomena has found numerous applications in many areas of the physical and chemical sciences, and this paper reviews the extensive experiments. Other subjects discussed include quantum optimal control theory, quantum control landscapes, the role of theoretical control designs in experimental realizations, and real-time quantum feedback control. The paper concludes with a prospective of open research directions that are likely to attract significant attention in the future.Comment: Review article, final version (significantly updated), 76 pages, accepted for publication in New J. Phys. (Focus issue: Quantum control

Additives to Control Mechanical Properties and Drug Delivery of Injectable Polymeric Scaffolds

In situ forming implants (ISIs) are popular due to their ease of use and local drug delivery potential, but they suffer from high initial drug burst, and release behavior is tied closely to solvent exchange and polymer properties. Additionally, such systems are traditionally viewed purely as drug delivery devices rather than potential scaffold materials due to their poor mechanical properties and minimal porosity. The aim of this research was to develop an injectable ISI with drug release, mechanical, and microstructural properties controlled by micro- and nanoparticle additives. First, an injectable ISI was developed with appropriate drug release kinetics for orthopedic applications. Poly(β-amino ester) (PBAE) microparticles were loaded with simvastatin or clodronate, and their loading efficiency and drug retention after washing was quantified. Drug-loaded PBAE microparticles and hydroxyapatite (HA) microparticles were added to a poly(lactic-co-glycolic acid) (PLGA)–based ISI. By loading simvastatin into PBAE microparticles, release was extended from 10 days to 30 days, and burst was reduced from 81% to 39%. Clodronate burst was reduced after addition of HA, but was unaffected by PBAE loading. Scaffold mass and porosity fluctuated as the scaffolds swelled and then degraded over 40 days. Next, the mechanical properties of these composite ISIs were quantified. Both micro- and nanoparticulate HA as well as PBAE microparticle content were varied. Increasing HA content generally improved compressive strength and modulus, with a plateau occurring at 30% nano-HA. Injectability remained clinically acceptable for up to 10% w/w PBAE microparticles. Ex vivo injections into trabecular bone improved both strength and modulus. Lastly, HA-free ISIs were investigated for drug delivery into the gingiva to treat periodontitis. Doxycycline and simvastatin were co-delivered, with delivery of doxycycline over 1 week accompanied by simvastatin release over 30 days. PBAE-containing ISIs exhibited higher initial and progressive porosity and accessible volume than PBAE-free ISIs over the course of degradation. Additionally, PBAE-containing ISIs provided superior tissue retention within a simulated periodontal pocket. The ISIs investigated here have a wide range of potential applications due to their flexible material and drug release properties, which can be controlled by both the chemistry and concentration of various particulate additives

In vitro modelling of tongue derived microbial biofilms and their reponse to treatment

This work concerns the development of a flat plate perfusion model to study biofilms derived from human tongue biota. The model has been derived from a previous sorbarod model, via a flat plate model (used to study wound organisms), to the model described in this thesis. The specific technical objectives were; 1. To measure biofilm pH in real time, 2. To extend VOC analysis by SIFT-MS to six biofilms in parallel and 3. To enable photodynamic interventions and optical monitoring of bioluminescent and non-bioluminescent organisms. The specific scientific objectives were; 1. To validate the model by comparison of in vivo and in vitro case studies, 2. To characterise the in vivo biofilm ecology and compare with ecology in vitro, 3. To compare existing and novel anti-malodour preparations and biofilm disrupting agents (including D-amino acids) and 4. To assess and aid the development of a novel handheld surface plasmon resonance based device for measuring oral volatile compounds.The results demonstrated that the biofilms transplanted from human donors are stable and reproducible, and that profiles of volatile compounds are retained in the transplanted biofilm, with high and low malodour individuals producing high and low malodour biofilms (profiles are indistinguishable by χ2 analysis at p < 0.1). The model was used to evaluate a novel formulation which was shown to be more effective than similar active compounds and controls (p < 0.05). In a further experiment, exposure of biofilms to D-amino acids during the growth phase was shown to cause significant (P< 0.05) effects onmicrobial and EPS composition compared with controls.Finally, the model in conjunction with SIFT-MS has been used to assess the performance of a novel surface plasmon resonance based biosensor. This biosensor has been shown to distinguish high and low malodour biofilms both in vitro and in vivo.In conclusion it has been demonstrated that the flat plate perfusion system is a stable, reproducible and accurate model covering many of the main aspects of a real tongue biofilm, and it has many advantages when compared with other published biofilm models