97 research outputs found
The Universal Statistical Structure and Scaling Laws of Chaos and Turbulence
Turbulence is a complex spatial and temporal structure created by the strong
non-linear dynamics of fluid flows at high Reynolds numbers. Despite being an
ubiquitous phenomenon that has been studied for centuries, a full understanding
of turbulence remained a formidable challenge. Here, we introduce tools from
the fields of quantum chaos and Random Matrix Theory (RMT) and present a
detailed analysis of image datasets generated from turbulence simulations of
incompressible and compressible fluid flows. Focusing on two observables: the
data Gram matrix and the single image distribution, we study both the local and
global eigenvalue statistics and compare them to classical chaos, uncorrelated
noise and natural images. We show that from the RMT perspective, the turbulence
Gram matrices lie in the same universality class as quantum chaotic rather than
integrable systems, and the data exhibits power-law scalings in the bulk of its
eigenvalues which are vastly different from uncorrelated classical chaos,
random data, natural images. Interestingly, we find that the single sample
distribution only appears as fully RMT chaotic, but deviates from chaos at
larger correlation lengths, as well as exhibiting different scaling properties.Comment: 9 pages, 4 figure
The Underlying Scaling Laws and Universal Statistical Structure of Complex Datasets
We study universal traits which emerge both in real-world complex datasets,
as well as in artificially generated ones. Our approach is to analogize data to
a physical system and employ tools from statistical physics and Random Matrix
Theory (RMT) to reveal their underlying structure. We focus on the
feature-feature covariance matrix, analyzing both its local and global
eigenvalue statistics. Our main observations are: (i) The power-law scalings
that the bulk of its eigenvalues exhibit are vastly different for uncorrelated
random data compared to real-world data, (ii) this scaling behavior can be
completely recovered by introducing long range correlations in a simple way to
the synthetic data, (iii) both generated and real-world datasets lie in the
same universality class from the RMT perspective, as chaotic rather than
integrable systems, (iv) the expected RMT statistical behavior already
manifests for empirical covariance matrices at dataset sizes significantly
smaller than those conventionally used for real-world training, and can be
related to the number of samples required to approximate the population
power-law scaling behavior, (v) the Shannon entropy is correlated with local
RMT structure and eigenvalues scaling, and substantially smaller in strongly
correlated datasets compared to uncorrelated synthetic data, and requires fewer
samples to reach the distribution entropy. These findings can have numerous
implications to the characterization of the complexity of data sets, including
differentiating synthetically generated from natural data, quantifying noise,
developing better data pruning methods and classifying effective learning
models utilizing these scaling laws.Comment: 16 pages, 7 figure
The Supercooling Window at Weak and Strong Coupling
Supercooled first order phase transitions are typical of theories where
conformal symmetry is predominantly spontaneously broken. In these theories the
fate of the flat scalar direction is highly sensitive to the size and the
scaling dimension of the explicit breaking deformations. For a given
deformation, the coupling must lie in a particular region to realize a
supercooled first order phase transition. We identify the supercooling window
in weakly coupled theories and derive a fully analytical understanding of its
boundaries. Mapping these boundaries allows us to identify the deformations
enlarging the supercooling window and to characterize their dynamics
analytically. For completeness we also discuss strongly coupled conformal field
theories with an holographic dual, where the complete characterization of the
supercooling window is challenged by calculability issues.Comment: 16 pages + appendices, 12 figures; v2: minor typo correcte
Grokking in Linear Estimators -- A Solvable Model that Groks without Understanding
Grokking is the intriguing phenomenon where a model learns to generalize long
after it has fit the training data. We show both analytically and numerically
that grokking can surprisingly occur in linear networks performing linear tasks
in a simple teacher-student setup with Gaussian inputs. In this setting, the
full training dynamics is derived in terms of the training and generalization
data covariance matrix. We present exact predictions on how the grokking time
depends on input and output dimensionality, train sample size, regularization,
and network initialization. We demonstrate that the sharp increase in
generalization accuracy may not imply a transition from "memorization" to
"understanding", but can simply be an artifact of the accuracy measure. We
provide empirical verification for our calculations, along with preliminary
results indicating that some predictions also hold for deeper networks, with
non-linear activations.Comment: 17 pages, 6 figure
Noise Injection as a Probe of Deep Learning Dynamics
We propose a new method to probe the learning mechanism of Deep Neural
Networks (DNN) by perturbing the system using Noise Injection Nodes (NINs).
These nodes inject uncorrelated noise via additional optimizable weights to
existing feed-forward network architectures, without changing the optimization
algorithm. We find that the system displays distinct phases during training,
dictated by the scale of injected noise. We first derive expressions for the
dynamics of the network and utilize a simple linear model as a test case. We
find that in some cases, the evolution of the noise nodes is similar to that of
the unperturbed loss, thus indicating the possibility of using NINs to learn
more about the full system in the future.Comment: 11 pages, 3 figure
Noise Injection Node Regularization for Robust Learning
We introduce Noise Injection Node Regularization (NINR), a method of
injecting structured noise into Deep Neural Networks (DNN) during the training
stage, resulting in an emergent regularizing effect. We present theoretical and
empirical evidence for substantial improvement in robustness against various
test data perturbations for feed-forward DNNs when trained under NINR. The
novelty in our approach comes from the interplay of adaptive noise injection
and initialization conditions such that noise is the dominant driver of
dynamics at the start of training. As it simply requires the addition of
external nodes without altering the existing network structure or optimization
algorithms, this method can be easily incorporated into many standard problem
specifications. We find improved stability against a number of data
perturbations, including domain shifts, with the most dramatic improvement
obtained for unstructured noise, where our technique outperforms other existing
methods such as Dropout or regularization, in some cases. We further show
that desirable generalization properties on clean data are generally
maintained.Comment: 16 pages, 9 figure
Oral White Lesions Associated with Chewing Khat
<p>Abstract</p> <p>Introduction</p> <p>Khat is a cultivated plant whose leaves when chewed elevate mood. Unlike the chewing of betel nut, no association between the white oral mucosal lesions in khat users and oral malignancies has been reported. Chewing of khat has been documented in many countries and has increased with worldwide migration. The impact of chewing khat upon the oral mucosa is essentially unknown.</p> <p>Purpose</p> <p>The purpose of this study was to assess the occurrence of oral white changes in chronic khat chewers. Oral mucosal changes in a group of 47 Yemenite Israeli men over 30 years of age, who had chewed khat more than 3 years, were compared to those of 55 Yemenite men who did not chew.</p> <p>Results</p> <p>White lesions were significantly more prevalent in the khat chewers (83%) compared to the non chewing individuals (16%) (P < 0.001). White oral lesions were identified primarily on the lower buccal attached gingival mucosa, the alveolar mucosa and the lower mucobuccal fold on the chewing side (p < 0.001). There was no significant association between the occurrence of the white lesions and smoking. Even though the majority of the white lesions (85.4%) were homogenous, 71.4% of the non homogenous lesions were identified in khat chewers. Vital staining with toluidine blue and exfoliative cytology was conducted on a subset of patients with homogenous and non-homogenous oral lesions, and there were no findings suspicious for pre-malignant or malignant changes.</p> <p>Discussion</p> <p>This study demonstrated a relationship between khat chewing and oral white lesions, which we attribute to chronic local mechanical and chemical irritation of the mucosa. Our findings also suggest that mucosal changes associated with khat are benign, however, this initial study requires further studies including follow-up of khat users to confirm the current findings, including the likely benign changes associated with chronic use and histologic findings of clinical lesions.</p
Charting the Topography of the Neural Network Landscape with Thermal-Like Noise
The training of neural networks is a complex, high-dimensional, non-convex
and noisy optimization problem whose theoretical understanding is interesting
both from an applicative perspective and for fundamental reasons. A core
challenge is to understand the geometry and topography of the landscape that
guides the optimization. In this work, we employ standard Statistical Mechanics
methods, namely, phase-space exploration using Langevin dynamics, to study this
landscape for an over-parameterized fully connected network performing a
classification task on random data. Analyzing the fluctuation statistics, in
analogy to thermal dynamics at a constant temperature, we infer a clear
geometric description of the low-loss region. We find that it is a
low-dimensional manifold whose dimension can be readily obtained from the
fluctuations. Furthermore, this dimension is controlled by the number of data
points that reside near the classification decision boundary. Importantly, we
find that a quadratic approximation of the loss near the minimum is
fundamentally inadequate due to the exponential nature of the decision boundary
and the flatness of the low-loss region. This causes the dynamics to sample
regions with higher curvature at higher temperatures, while producing
quadratic-like statistics at any given temperature. We explain this behavior by
a simplified loss model which is analytically tractable and reproduces the
observed fluctuation statistics.Comment: 7 pages, 4 figure
Rhizobacteria Impact Colonization of Listeria monocytogenes on Arabidopsis thaliana Roots
In spite of its relevance as a foodborne pathogen, we have limited knowledge about Listeria monocytogenes in the environment. L. monocytogenes outbreaks have been linked to fruits and vegetables; thus, a better understanding of the factors influencing its ability to colonize plants is important. We tested how environmental factors and other soil- and plant-associated bacteria influenced L. monocytogenes\u27 ability to colonize plant roots using Arabidopsis thaliana seedlings in a hydroponic growth system. We determined that the successful root colonization of L. monocytogenes 10403S was modestly but significantly enhanced by the bacterium being pregrown at higher temperatures, and this effect was independent of the biofilm and virulence regulator PrfA. We tested 14 rhizosphere-derived bacteria for their impact on L. monocytogenes 10403S, identifying one that enhanced and 10 that inhibited the association of 10403S with plant roots. We also characterized the outcomes of these interactions under both coinoculation and invasion conditions. We characterized the physical requirements of five of these rhizobacteria to impact the association of L. monocytogenes 10403S with roots, visualizing one of these interactions by microscopy. Furthermore, we determined that two rhizobacteria (one an inhibitor, the other an enhancer of 10403S root association) were able to similarly impact 10 different L. monocytogenes strains, indicating that the effects of these rhizobacteria on L. monocytogenes are not strain specific. Taken together, our results advance our understanding of the parameters that affect L. monocytogenes plant root colonization, knowledge that may enable us to deter its association with and, thus, downstream contamination of, food crops.
IMPORTANCE Listeria monocytogenes is ubiquitous in the environment, being found in or on soil, water, plants, and wildlife. However, little is known about the requirements for L. monocytogenes\u27 existence in these settings. Recent L. monocytogenes outbreaks have been associated with contaminated produce; thus, we used a plant colonization model to investigate factors that alter L. monocytogenes\u27 ability to colonize plant roots. We show that L. monocytogenes colonization of roots was enhanced when grown at higher temperatures prior to inoculation but did not require a known regulator of virulence and biofilm formation. Additionally, we identified several rhizobacteria that altered the ability of 11 different strains of L. monocytogenes to colonize plant roots. Understanding the factors that impact L. monocytogenes physiology and growth will be crucial for finding mechanisms (whether chemical or microbial) that enable its removal from plant surfaces to reduce L. monocytogenes contamination of produce and eliminate foodborne illness
- …