BPGrad: Towards Global Optimality in Deep Learning via Branch and Pruning

Understanding the global optimality in deep learning (DL) has been attracting more and more attention recently. Conventional DL solvers, however, have not been developed intentionally to seek for such global optimality. In this paper we propose a novel approximation algorithm, BPGrad, towards optimizing deep models globally via branch and pruning. Our BPGrad algorithm is based on the assumption of Lipschitz continuity in DL, and as a result it can adaptively determine the step size for current gradient given the history of previous updates, wherein theoretically no smaller steps can achieve the global optimality. We prove that, by repeating such branch-and-pruning procedure, we can locate the global optimality within finite iterations. Empirically an efficient solver based on BPGrad for DL is proposed as well, and it outperforms conventional DL solvers such as Adagrad, Adadelta, RMSProp, and Adam in the tasks of object recognition, detection, and segmentation

Squaraine Dyes, Design And Synthesis For Various Functional Materials Applications

This dissertation contains the synthesis and characterization of squaraine based new functional materials. In the first part of this thesis work, a water soluble benzothiazolium squaraine dye was synthesized with pyridium pendents, and controlled aggregation properties were achieved. After formation of partially reversible J-aggregation on a polyelectrolyte (poly(acryl acid) sodium salt) template, the nonlinear, two-photon absorption cross section per repeat unit was found to be above 30-fold enhanced compared with nonaggregate and/or low aggregates. Using a similar strategy, sulfonate anions were introduced into the squaraine structure, and the resulting compounds exhibited good water solubilities. A ‘turn on’ fluorescence was discovered when these squaraine dyes interacted with bovine serum albumin (BSA), titration studies by BSA site selective reagents show these squaraine dyes can bind to both site I and II of BSA, with a preference of site II. Introduction of these squaraine dyes to BSA nanoparticles generated near-IR protein nano fabricates, and cell images were collected. Metal sensing properties were also studied using the sulfonates containing a benzoindolium squaraine dye, and the linear response of the absorption of the squaraine dye to the concentration of Hg2+ makes it a good heavy metal-selective sensing material that can be carried out in aqueous solution. Later, a squaraine scaffold was attached to deoxyribonucleosides by Sonogashira coupling reactions, in which the reaction conditions were modified. Iodo-deoxyuridine and bromo-deoxyadenosine were used as the deoxyribonucleosides building blocks, and the resulting squaraine dye-modified deoxyribonucleosides exhibited near-IR absorption and emission properties due to the squaraine chromophore. Interestingly, these non-natural deoxyribonucleosdies showed viscosity dependent photophysical properties, which make them nice candidates for fluorescence viscosity sensors at the cellular level. After incubation with cells, these iv viscosity sensors were readily uptaken by cell, and images were obtained showing regions of high viscosity in cells