A Control Theoretic Framework for Adaptive Gradient Optimizers in Machine Learning

Adaptive gradient methods have become popular in optimizing deep neural networks; recent examples include AdaGrad and Adam. Although Adam usually converges faster, variations of Adam, for instance, the AdaBelief algorithm, have been proposed to enhance Adam's poor generalization ability compared to the classical stochastic gradient method. This paper develops a generic framework for adaptive gradient methods that solve non-convex optimization problems. We first model the adaptive gradient methods in a state-space framework, which allows us to present simpler convergence proofs of adaptive optimizers such as AdaGrad, Adam, and AdaBelief. We then utilize the transfer function paradigm from classical control theory to propose a new variant of Adam, coined AdamSSM. We add an appropriate pole-zero pair in the transfer function from squared gradients to the second moment estimate. We prove the convergence of the proposed AdamSSM algorithm. Applications on benchmark machine learning tasks of image classification using CNN architectures and language modeling using LSTM architecture demonstrate that the AdamSSM algorithm improves the gap between generalization accuracy and faster convergence than the recent adaptive gradient methods

Iteratively Preconditioned Gradient-Descent Approach for Moving Horizon Estimation Problems

Moving horizon estimation (MHE) is a widely studied state estimation approach in several practical applications. In the MHE problem, the state estimates are obtained via the solution of an approximated nonlinear optimization problem. However, this optimization step is known to be computationally complex. Given this limitation, this paper investigates the idea of iteratively preconditioned gradient-descent (IPG) to solve MHE problem with the aim of an improved performance than the existing solution techniques. To our knowledge, the preconditioning technique is used for the first time in this paper to reduce the computational cost and accelerate the crucial optimization step for MHE. The convergence guarantee of the proposed iterative approach for a class of MHE problems is presented. Additionally, sufficient conditions for the MHE problem to be convex are also derived. Finally, the proposed method is implemented on a unicycle localization example. The simulation results demonstrate that the proposed approach can achieve better accuracy with reduced computational costs

MTA - The Barrier for Endodontic Success!

The elimination of microbiological colonization in any parts of human body is a crucial principle of treating infection  and root canal system is no exception. The major challenges associated with endodontic treatment of teeth with open apices are achieving complete debridement, canal disinfection and optimal sealing of the root canal system.In the absence of a natural apical constriction, the production of mineralized tissue in the apical region is important to create an apical barrier and allow 3-dimensional adaptation of obturating material within the root canal system.A clinical case of central incisor have been treated with the use of an apical plug of MTA for apexification that had suffered premature interruption of root development as a consequence of trauma.The remaining portion of the root canals was then closed with thermoplasticized gutta-percha. At 6-month follow-up period the clinical and radiographic appearance of the tooth showed resolution of the periapical lesions