Grokking in Linear Estimators -- A Solvable Model that Groks without Understanding

Grokking is the intriguing phenomenon where a model learns to generalize long after it has fit the training data. We show both analytically and numerically that grokking can surprisingly occur in linear networks performing linear tasks in a simple teacher-student setup with Gaussian inputs. In this setting, the full training dynamics is derived in terms of the training and generalization data covariance matrix. We present exact predictions on how the grokking time depends on input and output dimensionality, train sample size, regularization, and network initialization. We demonstrate that the sharp increase in generalization accuracy may not imply a transition from "memorization" to "understanding", but can simply be an artifact of the accuracy measure. We provide empirical verification for our calculations, along with preliminary results indicating that some predictions also hold for deeper networks, with non-linear activations.Comment: 17 pages, 6 figure

Markov Chain Methods For Analyzing Complex Transport Networks

We have developed a steady state theory of complex transport networks used to model the flow of commodity, information, viruses, opinions, or traffic. Our approach is based on the use of the Markov chains defined on the graph representations of transport networks allowing for the effective network design, network performance evaluation, embedding, partitioning, and network fault tolerance analysis. Random walks embed graphs into Euclidean space in which distances and angles acquire a clear statistical interpretation. Being defined on the dual graph representations of transport networks random walks describe the equilibrium configurations of not random commodity flows on primary graphs. This theory unifies many network concepts into one framework and can also be elegantly extended to describe networks represented by directed graphs and multiple interacting networks.Comment: 26 pages, 4 figure

Functional limit theorems for random regular graphs

Consider d uniformly random permutation matrices on n labels. Consider the sum of these matrices along with their transposes. The total can be interpreted as the adjacency matrix of a random regular graph of degree 2d on n vertices. We consider limit theorems for various combinatorial and analytical properties of this graph (or the matrix) as n grows to infinity, either when d is kept fixed or grows slowly with n. In a suitable weak convergence framework, we prove that the (finite but growing in length) sequences of the number of short cycles and of cyclically non-backtracking walks converge to distributional limits. We estimate the total variation distance from the limit using Stein's method. As an application of these results we derive limits of linear functionals of the eigenvalues of the adjacency matrix. A key step in this latter derivation is an extension of the Kahn-Szemer\'edi argument for estimating the second largest eigenvalue for all values of d and n.Comment: Added Remark 27. 39 pages. To appear in Probability Theory and Related Field

Motor corticospinal excitability: a novel facet of pain modulation?

Abstract. Introduction:. Increase in excitability of the primary motor cortex (M1) is associated with pain inhibition by analgesics, which is, in turn, associated with the psychophysical antinociceptive pain modulation profile. However, the relationship between neurophysiological M1 excitability and psychophysical pain modulation has not yet been explored. Objectives:. We aim to study these relationships in healthy subjects. Methods:. Forty-one young healthy subjects (22 women) underwent a wide battery of psychophysical testing that included conditioned pain modulation (CPM) and pain temporal summation, and a transcranial magnetic stimulation neurophysiological assessment of the motor corticospinal excitability, including resting motor threshold, motor-evoked potentials (MEPs), and cortical silent period. Results:. Increased motor corticospinal excitability in 2 parameters was associated with more efficient CPM: (1) higher MEP amplitude (r = −0.574; P_Bonferroni = 0.02) and (2) longer MEP duration (r = −0.543; P_Bonferroni = 0.02). The latter also correlated with the lower temporal summation magnitude (r = −0.421; P = 0.007); however, on multiplicity adjustment, significance was lost. Conclusions:. Increased corticospinal excitability of the primary motor cortex is associated with more efficient inhibitory pain modulation as assessed by CPM, in healthy subjects. Motor-evoked potential amplitude and duration may be considered as an additional, objective and easy to measure parameter to allow for better individual assessment of pain modulation profile

MRI-Guided Focused Ultrasound in Parkinson’s Disease: A Review

MRI-guided focused ultrasound is a new technology that enables intracranial ablation. Since lesioning ameliorates some of the symptoms of PD, this technology is being explored as a possible treatment for medication resistant symptoms in PD patients. The purpose of this paper is to review the clinical use and treatment outcomes of PD patients treated to date with this technology

The Role of the Anesthesiologist during Magnetic Resonance-Guided Focused Ultrasound Thalamotomy for Tremor: A Single-Center Experience

Ablative incisionless neurosurgery has become possible through advances in focused ultrasound and magnetic resonance imaging (MRI). The great advantage of MRI-guided focused ultrasound (MRgFUS) is that the ablation is performed through an intact skull without surgery. Here, we review the new modality of MRgFUS for treating tremor and enlighten the role of the anesthesiologist in the unique procedural setting of the MRI suite. During the MRgFUS process, the patients should be awake and are required to cooperate with the medical staff to allow assessment of tremor reduction and potential occurrence of adverse effects. In addition, the patient’s head is immobilized inside the MRI tunnel for hours. This combination presents major challenges for the attending anesthesiologist, who is required to try to prevent pain and nausea and when present, to treat these symptoms. Anxiety, vertigo, and vomiting may occur during treatment and require urgent treatment. Here, we review the literature available on anesthetic management during the procedure and our own experience and provide recommendations based on our collected knowledge

Real-time change detection of steady-state evoked potentials

Steady-state evoked potentials (SSEP) are the electrical activity recorded from the scalp in response to high-rate sensory stimulation. SSEP consist of a constituent frequency component matching the stimulation rate, whose amplitude and phase remain constant with time and are sensitive to functional changes in the stimulated sensory system. Monitoring SSEP during neurosurgical procedures allows identification of an emerging impairment early enough before the damage becomes permanent. In routine practice, SSEP are extracted by averaging of the EEG recordings, allowing detection of neurological changes within approximately a minute. As an alternative to the relatively slow-responding empirical averaging, we present an algorithm that detects changes in the SSEP within seconds. Our system alerts when changes in the SSEP are detected by applying a two-step Generalized Likelihood Ratio Test (GLRT) on the unaveraged EEG recordings. This approach outperforms conventional detection and provides the monitor with a statistical measure of the likelihood that a change occurred, thus enhancing its sensitivity and reliability. The system’s performance is analyzed using Monte Carlo simulations and tested on real EEG data recorded under coma

Focused Ultrasound Thalamotomy for Tremor Relief in Atypical Parkisnsonism

Background. Magnetic resonance imaging (MRI)-guided focused ultrasound (FUS) VIM-thalamotomy has established efficacy and safety in tremor relief in patients with essential tremor and Parkinson’s disease. The efficacy and safety in patients with atypical parkinsonism have not been reported. Objective. To report on the efficacy and safety of FUS VIM-thalamotomy in 8 patients with parkinsonism, multiple system atrophy-Parkinsonian type (MSA-P) (n = 5), and dementia with Lewy bodies (DLB) (n = 3). Methods. Tremor was assessed in the treated hemibody using the Clinical Rating Scale for Tremor (CRST). The motor Unified MSA Rating Scale (UMSAR) was used in the MSA-P and motor sections of the Unified Parkinson’s Disease Rating Scale (UPDRS-III) in DLB patients. Cognition was measured using the Montreal Cognitive Assessment (MoCA). Results. In MSA-P and DLB patients, there was immediate tremor relief. CRST scores measured on the treated side improved compared to baseline. During the follow-up of up to 1 year tremor reduction persisted. The change in CRST scores at different time points did not reach statistical significance, probably due to the small sample size. Adverse events were transient and resolved within a year. Conclusions. In our experience, FUS VIM-thalamotomy was effective in patients with MSA-P and DLB. Larger, controlled studies are needed to verify our preliminary observations

Trace Elements in Tears: Comparison of Rural and Urban Populations Using Particle Induced X-ray Emission

We aimed to evaluate the types and concentrations of trace elements in tears of individuals living in urban and rural environments using particle induced X-ray emission (PIXE) and the possible association with exposure to air pollution and suggest a novel method for tear-based biomonitoring studies. This cross-sectional pilot study comprised 42 healthy subjects, 28 living in a rural area and 14 in an industrial city. Tears were collected with Schirmer paper and characterized by PIXE. Trace element concentrations from both eyes were averaged together with environmental pollution data. Main outcome measures were between-group differences in types and concentrations of trace elements in tears and comparison to environmental data. The rural group included 12/28 men, mean age 45.2 ± 14.8 years. The urban group consisted of 11/14 men of mean age 27 ± 5.9 years. Six rural and all urban were active smokers. Air pollution data showed more toxic elements in the rural environment. On PIXE analysis, chlorine, sodium, and potassium were found in similar concentrations in all samples. Normalizing to chlorine yielded higher values of aluminum, iron, copper, and titanium in the rural group; aluminum was found only in the rural group. The higher levels of certain trace elements in the rural group may, in part, be a consequence of exposure to specific environmental conditions. No direct association was found with air pollution data. PIXE is useful to analyze trace elements in tears, which might serve as a marker for individual exposure to environmental pollutants in biomonitoring studies