Escaping Saddle Points with Adaptive Gradient Methods

Adaptive methods such as Adam and RMSProp are widely used in deep learning but are not well understood. In this paper, we seek a crisp, clean and precise characterization of their behavior in nonconvex settings. To this end, we first provide a novel view of adaptive methods as preconditioned SGD, where the preconditioner is estimated in an online manner. By studying the preconditioner on its own, we elucidate its purpose: it rescales the stochastic gradient noise to be isotropic near stationary points, which helps escape saddle points. Furthermore, we show that adaptive methods can efficiently estimate the aforementioned preconditioner. By gluing together these two components, we provide the first (to our knowledge) second-order convergence result for any adaptive method. The key insight from our analysis is that, compared to SGD, adaptive methods escape saddle points faster, and can converge faster overall to second-order stationary points.Comment: Update Theorem 4.1 and proof to use martingale concentration bounds, i.e. matrix Freedma

Jefferson and Tocqueville on Christian Nationalism

The United States’ longstanding principle of the separation of church and state has been put into question by proponents of Christian nationalism who seek to make the United States an established Christian state. This paper seeks to argue that Thomas Jefferson’s philosophy was based on sound reasoning, and its benefits to American society were later recognized by French philosopher Alexis de Tocqueville. Furthermore, this paper uses Jefferson’s philosophy and Tocqueville\u27s analysis and applies it to the contemporary relationship between church and state in the United States to argue that Christian nationalism is unfeasible, and that religion should remain a private matter where it provides the most benefits to American citizens

Genetic advances in dermatophytes

Millions of superficial fungal infections are annually observed in humans and animals. The majority of these mycoses are caused by dermatophytes, a specialized group of filamentous fungi that exclusively infect keratinized host structures. Despite the high prevalence of the disease, dermatophytosis, little is known about the pathogenicity mechanisms of these microorganisms. This drawback may be related to the fact that dermatophytes have been investigated poorly at the molecular level. In contrast to many other pathogenic fungi, they grow comparatively slowly under in vitro conditions, and in the last decades, only a limited number of molecular tools have been established for their manipulation. In recent years, however, major promising approaches were undertaken to improve genetic analyses in dermatophytes. These strategies include efficient systems for targeted gene inactivation and gene silencing, and broad transcriptional profiling techniques, which have even been applied in sophisticated infection models. As a fundamental prerequisite for future genetic analyses, full genome sequences of seven different dermatophyte species have become available recently. Therefore, it appeared timely to review the available molecular tools and methodologies in dermatophyte research, which may provide future insights into the virulence of these clinically important pathogen

Effect of adrenergic agonists on phosphoinositide breakdown in rat skeletal muscle preparations

AbstractAdrenergic regulation of phosphoinositide breakdown in rat skeletal muscle was investigated in 30-min incubations with 10 mM LiCl. In rat hemidiaphragms, prelabelled with D-myo-[2-3H]inositol, addition of α-agonists (epinephrine, norepinephrine, phenylephrine) induced a 5-8-fold increase of [3H]inositol monophosphate accumulation. This could be prevented by inclusion of α-antagonists (phentolamine, prazosin). β-Agonists and/or β-antagonists had no effect. Similar experiments with isolated flexor digitorum brevis muscle fibers yielded confirmatory results. Functional integrity of β-receptor mediated processes was suggested by the β-agonist-induced increase of glucose 6-phosphate in hemidiaphragms and cAMP in fiber preparations. The results indicate that phosphoinositide breakdown in differentiated rat skeletal muscle is, at least in part, under α-adrenergic control