Escaping Saddle Points with Adaptive Gradient Methods

Adaptive methods such as Adam and RMSProp are widely used in deep learning but are not well understood. In this paper, we seek a crisp, clean and precise characterization of their behavior in nonconvex settings. To this end, we first provide a novel view of adaptive methods as preconditioned SGD, where the preconditioner is estimated in an online manner. By studying the preconditioner on its own, we elucidate its purpose: it rescales the stochastic gradient noise to be isotropic near stationary points, which helps escape saddle points. Furthermore, we show that adaptive methods can efficiently estimate the aforementioned preconditioner. By gluing together these two components, we provide the first (to our knowledge) second-order convergence result for any adaptive method. The key insight from our analysis is that, compared to SGD, adaptive methods escape saddle points faster, and can converge faster overall to second-order stationary points.Comment: Update Theorem 4.1 and proof to use martingale concentration bounds, i.e. matrix Freedma

Effect of adrenergic agonists on phosphoinositide breakdown in rat skeletal muscle preparations

AbstractAdrenergic regulation of phosphoinositide breakdown in rat skeletal muscle was investigated in 30-min incubations with 10 mM LiCl. In rat hemidiaphragms, prelabelled with D-myo-[2-3H]inositol, addition of α-agonists (epinephrine, norepinephrine, phenylephrine) induced a 5-8-fold increase of [3H]inositol monophosphate accumulation. This could be prevented by inclusion of α-antagonists (phentolamine, prazosin). β-Agonists and/or β-antagonists had no effect. Similar experiments with isolated flexor digitorum brevis muscle fibers yielded confirmatory results. Functional integrity of β-receptor mediated processes was suggested by the β-agonist-induced increase of glucose 6-phosphate in hemidiaphragms and cAMP in fiber preparations. The results indicate that phosphoinositide breakdown in differentiated rat skeletal muscle is, at least in part, under α-adrenergic control

Oestrogen-Binding Components in Human Renal Cell Carcinoma

Peer Reviewe

A comprehensive ovine model of blood transfusion

Background: The growing awareness of transfusion-associated morbidity and mortality necessitates investigations into the underlying mechanisms. Small animals have been the dominant transfusion model but have associated limitations. This study aimed to develop a comprehensive large animal (ovine) model of transfusion encompassing: blood collection, processing and storage, compatibility testing right through to post-transfusion outcomes. Materials and methods: Two units of blood were collected from each of 12 adult male Merino sheep and processed into 24 ovine-packed red blood cell (PRBC) units. Baseline haematological parameters of ovine blood and PRBC cells were analysed. Biochemical changes in ovine PRBCs were characterized during the 42-day storage period. Immunological compatibility of the blood was confirmed with sera from potential recipient sheep, using a saline and albumin agglutination cross-match. Following confirmation of compatibility, each recipient sheep (n = 12) was transfused with two units of ovine PRBC. Results: Procedures for collecting, processing, cross-matching and transfusing ovine blood were established. Although ovine red blood cells are smaller and higher in number, their mean cell haemoglobin concentration is similar to human red blood cells. Ovine PRBC showed improved storage properties in saline-adenine-glucose-mannitol (SAG-M) compared with previous human PRBC studies. Seventy-six compatibility tests were performed and 17·1% were incompatible. Only cross-match compatible ovine PRBC were transfused and no adverse reactions were observed. Conclusion: These findings demonstrate the utility of the ovine model for future blood transfusion studies and highlight the importance of compatibility testing in animal models involving homologous transfusions

Relationship between quantum decoherence times and solvation dynamics in condensed phase chemical systems

A relationship between the time scales of quantum coherence loss and short-time solvent response for a solute/bath system is derived for a Gaussian wave packet approximation for the bath. Decoherence and solvent response times are shown to be directly proportional to each other, with the proportionality coefficient given by the ratio of the thermal energy fluctuations to the fluctuations in the system-bath coupling. The relationship allows the prediction of decoherence times for condensed phase chemical systems from well developed experimental methods.Comment: 10 pages, no figures, late

Ctrl-P:Temporal control of prosodic variation for speech synthesis

Text does not fully specify the spoken form, so text-to-speech models must be able to learn from speech data that vary in ways not explained by the corresponding text. One way to reduce the amount of unexplained variation in training data is to provide acoustic information as an additional learning signal. When generating speech, modifying this acoustic information enables multiple distinct renditions of a text to be produced. Since much of the unexplained variation is in the prosody, we propose a model that generates speech explicitly conditioned on the three primary acoustic correlates of prosody: $F_{0}$, energy and duration. The model is flexible about how the values of these features are specified: they can be externally provided, or predicted from text, or predicted then subsequently modified. Compared to a model that employs a variational auto-encoder to learn unsupervised latent features, our model provides more interpretable, temporally-precise, and disentangled control. When automatically predicting the acoustic features from text, it generates speech that is more natural than that from a Tacotron 2 model with reference encoder. Subsequent human-in-the-loop modification of the predicted acoustic features can significantly further increase naturalness.Comment: To be published in Interspeech 2021. 5 pages, 4 figure

Aquatics reconstruction software: the design of a diagnostic tool based on computer vision algorithms

Computer vision methods can be applied to a variety of medical and surgical applications, and many techniques and algorithms are available that can be used to recover 3D shapes and information from images range and volume data. Complex practical applications, however, are rarely approachable with a single technique, and require detailed analysis on how they can be subdivided in subtasks that are computationally treatable and that, at the same time, allow for the appropriate level of user-interaction. In this paper we show an example of a complex application where, following criteria of efficiency, reliability and user friendliness, several computer vision techniques have been selected and customized to build a system able to support diagnosis and endovascular treatment of Abdominal Aortic Aneurysms. The system reconstructs the geometrical representation of four different structures related to the aorta (vessel lumen, thrombus, calcifications and skeleton) from CT angiography data. In this way it supports the three dimensional measurements required for a careful geometrical evaluation of the vessel, that is fundamental to decide if the treatment is necessary and to perform, in this case, its planning. The system has been realized within the European trial AQUATICS (IST-1999-20226 EUTIST-M WP 12), and it has been widely tested on clinical data

Building and Testing a Statistical Shape Model of the Human Ear Canal

Abstract. Today the design of custom in-the-ear hearing aids is based on personal experience and skills and not on a systematic description of the variation of the shape of the ear canal. In this paper it is described how a dense surface point distribution model of the human ear canal is built based on a training set of laser scanned ear impressions and a sparse set of anatomical landmarks placed by an expert. The landmarks are used to warp a template mesh onto all shapes in the training set. Using the vertices from the warped meshes, a 3D point distribution model is made. The model is used for testing for gender related differences in size and shape of the ear canal.