157 research outputs found
Good regularity creates large learning rate implicit biases: edge of stability, balancing, and catapult
Large learning rates, when applied to gradient descent for nonconvex
optimization, yield various implicit biases including the edge of stability
(Cohen et al., 2021), balancing (Wang et al., 2022), and catapult (Lewkowycz et
al., 2020). These phenomena cannot be well explained by classical optimization
theory. Though significant theoretical progress has been made in understanding
these implicit biases, it remains unclear for which objective functions would
they be more likely. This paper provides an initial step in answering this
question and also shows that these implicit biases are in fact various tips of
the same iceberg. To establish these results, we develop a global convergence
theory under large learning rates, for a family of nonconvex functions without
globally Lipschitz continuous gradient, which was typically assumed in existing
convergence analysis. Specifically, these phenomena are more likely to occur
when the optimization objective function has good regularity. This regularity,
together with gradient descent using a large learning rate that favors flatter
regions, results in these nontrivial dynamical behaviors. Another corollary is
the first non-asymptotic convergence rate bound for large-learning-rate
gradient descent optimization of nonconvex functions. Although our theory only
applies to specific functions so far, the possibility of extrapolating it to
neural networks is also experimentally validated, for which different choices
of loss, activation functions, and other techniques such as batch normalization
can all affect regularity significantly and lead to very different training
dynamics
Score Matching-based Pseudolikelihood Estimation of Neural Marked Spatio-Temporal Point Process with Uncertainty Quantification
Spatio-temporal point processes (STPPs) are potent mathematical tools for
modeling and predicting events with both temporal and spatial features. Despite
their versatility, most existing methods for learning STPPs either assume a
restricted form of the spatio-temporal distribution, or suffer from inaccurate
approximations of the intractable integral in the likelihood training
objective. These issues typically arise from the normalization term of the
probability density function. Moreover, current techniques fail to provide
uncertainty quantification for model predictions, such as confidence intervals
for the predicted event's arrival time and confidence regions for the event's
location, which is crucial given the considerable randomness of the data. To
tackle these challenges, we introduce SMASH: a Score MAtching-based
pSeudolikeliHood estimator for learning marked STPPs with uncertainty
quantification. Specifically, our framework adopts a normalization-free
objective by estimating the pseudolikelihood of marked STPPs through
score-matching and offers uncertainty quantification for the predicted event
time, location and mark by computing confidence regions over the generated
samples. The superior performance of our proposed framework is demonstrated
through extensive experiments in both event prediction and uncertainty
quantification
Sample Complexity of Neural Policy Mirror Descent for Policy Optimization on Low-Dimensional Manifolds
Policy-based algorithms equipped with deep neural networks have achieved
great success in solving high-dimensional policy optimization problems in
reinforcement learning. However, current analyses cannot explain why they are
resistant to the curse of dimensionality. In this work, we study the sample
complexity of the neural policy mirror descent (NPMD) algorithm with
convolutional neural networks (CNN) as function approximators. Motivated by the
empirical observation that many high-dimensional environments have state spaces
possessing low-dimensional structures, such as those taking images as states,
we consider the state space to be a -dimensional manifold embedded in the
-dimensional Euclidean space with intrinsic dimension . We show that
in each iteration of NPMD, both the value function and the policy can be well
approximated by CNNs. The approximation errors are controlled by the size of
the networks, and the smoothness of the previous networks can be inherited. As
a result, by properly choosing the network size and hyperparameters, NPMD can
find an -optimal policy with
samples in expectation, where
indicates the smoothness of environment. Compared to previous
work, our result exhibits that NPMD can leverage the low-dimensional structure
of state space to escape from the curse of dimensionality, providing an
explanation for the efficacy of deep policy-based algorithms
Subband Independent Component Analysis for Coherence Enhancement
Objective: Cortico-muscular coherence (CMC) is becoming a common technique for detection and characterization of functional coupling between the motor cortex and muscle activity. It is typically evaluated between surface electromyogram (sEMG) and electroencephalogram (EEG) signals collected synchronously during controlled movement tasks. However, the presence of noise and activities unrelated to observed motor tasks in sEMG and EEG results in low CMC levels, which often makes functional coupling difficult to detect. Methods: In this paper, we introduce Coherent Subband Independent Component Analysis (CoSICA) to enhance synchronous cortico-muscular components in mixtures captured by sEMG and EEG. The methodology relies on filter bank processing to decompose sEMG and EEG signals into frequency bands. Then, it applies independent component analysis along with a component selection algorithm for re-synthesis of sEMG and EEG designed to maximize CMC levels. Results: We demonstrate the effectiveness of the proposed method in increasing CMC levels across different signal-to-noise ratios first using simulated data. Using neurophysiological data, we then illustrate that CoSICA processing achieves a pronounced enhancement of original CMC. Conclusion: Our findings suggest that the proposed technique provides an effective framework for improving coherence detection. Significance: The proposed methodologies will eventually contribute to understanding of movement control and has high potential for translation into clinical practice
Subband Independent Component Analysis for Coherence Enhancement
Objective: Cortico-muscular coherence (CMC) is becoming a common technique for detection and characterization of functional coupling between the motor cortex and muscle activity. It is typically evaluated between surface electromyo- gram (sEMG) and electroencephalogram (EEG) signals collected synchronously during controlled movement tasks. However, the presence of noise and activities unrelated to observed motor tasks in sEMG and EEG results in low CMC levels, which often makes functional coupling difficult to detect. Methods: In this paper, we introduce Coherent Subband Independent Component Analysis (CoSICA) to enhance synchronous cortico-muscular components in mixtures captured by sEMG and EEG. The methodology relies on filter bank processing to decompose sEMG and EEG signals into frequency bands. Then, it applies independent component analysis along with a component selection algorithm for re- synthesis of sEMG and EEG designed to maximize CMC levels. Results: We demonstrate the effectiveness of the proposed method in increasing CMC levels across different signal-to-noise ratios first using simulated data. Using neurophysiological data, we then illustrate that CoSICA processing achieves a pronounced enhancement of original CMC. Conclusion: Our findings suggest that the proposed technique provides an effective framework for improving coherence detection. Significance: The proposed methodologies will eventually contribute to understanding of movement control and has high potential for translation into clinical practice
- …