On the Convergence of WKB Approximations of the Damped Mathieu Equation

Consider the differential equation ${ m\ddot{x} +\gamma \dot{x} -x\epsilon \cos(\omega t) =0}$, $0 \leq t \leq T$. The form of the fundamental set of solutions are determined by Floquet theory. In the limit as $m \to 0$ we can apply WKB theory to get first order approximations of this fundamental set. WKB theory states that this approximation gets better as $m \to 0$ in the sense that the difference in sup norm is bounded as function of $m$ for a given $T$. However, convergence of the periodic parts and exponential parts are not addressed. We show that there is convergence to these components. The asymptotic error for the characteristic exponents are $O(m^2)$ and $O(m)$ for the periodic parts.Comment: 10 pages. version

Convergence Rates for Multi-classs Logistic Regression Near Minimum

In the current paper we provide constructive estimation of the convergence rate for training a known class of neural networks: the multi-class logistic regression. Despite several decades of successful use, our rigorous results appear new, reflective of the gap between practice and theory of machine learning. Training a neural network is typically done via variations of the gradient descent method. If a minimum of the loss function exists and gradient descent is used as the training method, we provide an expression that relates learning rate to the rate of convergence to the minimum. The method involves an estimate of the condition number of the Hessian of the loss function. We also discuss the existence of a minimum, as it is not automatic that a minimum exists. One method of ensuring convergence is by assigning positive probabiity to every class in the training dataset.Comment: minor changes in notation, theorem 7.1 fixe

Overdamped dynamics of a Brownian particle levitated in a Paul trap

We study the dynamics of the center of mass of a Brownian particle levitated in a Paul trap. We focus on the overdamped regime in the context of levitodynamics, comparing theory with our numerical simulations and experimental data from a nanoparticle in a Paul trap. We provide an exact analytical solution to the stochastic equation of motion, expressions for the standard deviation of the motion, and thermalization times by using the WKB method under two different limits. Finally, we prove the power spectral density of the motion can be approximated by that of an Ornstein-Uhlenbeck process and use the found expression to calibrate the motion of a trapped particle

Graph-based methods coupled with specific distributional distances for adversarial attack detection

Artificial neural networks are prone to being fooled by carefully perturbed inputs which cause an egregious misclassification. These \textit{adversarial} attacks have been the focus of extensive research. Likewise, there has been an abundance of research in ways to detect and defend against them. We introduce a novel approach of detection and interpretation of adversarial attacks from a graph perspective. For an image, benign or adversarial, we study how a neural network's architecture can induce an associated graph. We study this graph and introduce specific measures used to predict and interpret adversarial attacks. We show that graphs-based approaches help to investigate the inner workings of adversarial attacks

Neural Networks and Optical Character Recognition

The theme of this work is artificial neural networks. We discuss the mathematics of multi-class logistic regression, and secondly, we study the utility and limitations of a pure attention-based approach to optical character recognition (OCR). For multi-class logistic regression, we prove the existence of the minimum of the loss function when applied to gradient descent if the label matrix is fully smoothed. We also find bounds on the smallest and largest eigenvalues of the Hessian and compute its condition number. From the theory of numerical analysis, the condition number gives the maximum contraction rate possible using a learning rate parameter. For attention and OCR, we do experiments on isolated word recognition using a cursive font. These experiments show that attention relies excessively on memorization/correlation of letters, which is a limitation. It has serious trouble recognizing text when the training samples are significantly different from the test samples. This includes the case when the training data set consists of: bigrams; trigrams; random words

Graph-based methods coupled with specific distributional distances for adversarial attack detection

International audienceArtificial neural networks are prone to being fooled by carefully perturbed inputs which cause an egregious misclassification. These adversarial attacks have been the focus of extensive research. Likewise, there has been an abundance of research in ways to detect and defend against them. We introduce a novel approach of detection and interpretation of adversarial attacks from a graph perspective. For an input image, we compute an associated sparse graph using the layer-wise relevance propagation algorithm (Bach et al., 2015). Specifically, we only keep edges of the neural network with the highest relevance values. Three quantities are then computed from the graph which are then compared against those computed from the training set. The result of the comparison is a classification of the image as benign or adversarial. To make the comparison, two classification methods are introduced: (1) an explicit formula based on Wasserstein distance applied to the degree of node and (2) a logistic regression. Both classification methods produce strong results which lead us to believe that a graph-based interpretation of adversarial attacks is valuable

Exploring continual learning strategies in artificial neural networks through graph-based analysis of connectivity: insights from a brain-inspired perspective

Artificial Neural Networks (ANNs) aim at mimicking information processing in biological networks. In cognitive neuroscience, graph modeling is a powerful framework widely used to study brain structural and functional connectivity. Yet, the extension of graph modeling to ANNs has been poorly explored especially in term of functional connectivity (i.e. the contextual change of the activity's units in networks). From the perspective of designing more robust and interpretable ANNs, we study how a brain-inspired graph-based approach can be extended and used to investigate their properties and behaviors. We focus our study on different continual learning strategies inspired by the human brain and modeled with ANNs. We show that graph modeling offers a simple and elegant framework to deeply investigate ANNs, compare their performances and explore deleterious behaviors such as catastrophic forgetting

Exploring continual learning strategies in artificial neural networks through graph-based analysis of connectivity: insights from a brain-inspired perspective

Artificial Neural Networks (ANNs) aim at mimicking information processing in biological networks. In cognitive neuroscience, graph modeling is a powerful framework widely used to study brain structural and functional connectivity. Yet, the extension of graph modeling to ANNs has been poorly explored especially in term of functional connectivity (i.e. the contextual change of the activity's units in networks). From the perspective of designing more robust and interpretable ANNs, we study how a brain-inspired graph-based approach can be extended and used to investigate their properties and behaviors. We focus our study on different continual learning strategies inspired by the human brain and modeled with ANNs. We show that graph modeling offers a simple and elegant framework to deeply investigate ANNs, compare their performances and explore deleterious behaviors such as catastrophic forgetting

Exploring continual learning strategies in artificial neural networks through graph-based analysis of connectivity: insights from a brain-inspired perspective

Artificial Neural Networks (ANNs) aim at mimicking information processing in biological networks. In cognitive neuroscience, graph modeling is a powerful framework widely used to study brain structural and functional connectivity. Yet, the extension of graph modeling to ANNs has been poorly explored especially in term of functional connectivity (i.e. the contextual change of the activity's units in networks). From the perspective of designing more robust and interpretable ANNs, we study how a brain-inspired graph-based approach can be extended and used to investigate their properties and behaviors. We focus our study on different continual learning strategies inspired by the human brain and modeled with ANNs. We show that graph modeling offers a simple and elegant framework to deeply investigate ANNs, compare their performances and explore deleterious behaviors such as catastrophic forgetting

Exploring continual learning strategies in artificial neural networks through graph-based analysis of connectivity: insights from a brain-inspired perspective

Artificial Neural Networks (ANNs) aim at mimicking information processing in biological networks. In cognitive neuroscience, graph modeling is a powerful framework widely used to study brain structural and functional connectivity. Yet, the extension of graph modeling to ANNs has been poorly explored especially in term of functional connectivity (i.e. the contextual change of the activity's units in networks). From the perspective of designing more robust and interpretable ANNs, we study how a brain-inspired graph-based approach can be extended and used to investigate their properties and behaviors. We focus our study on different continual learning strategies inspired by the human brain and modeled with ANNs. We show that graph modeling offers a simple and elegant framework to deeply investigate ANNs, compare their performances and explore deleterious behaviors such as catastrophic forgetting