26,629 research outputs found
Annealing Optimization for Progressive Learning with Stochastic Approximation
In this work, we introduce a learning model designed to meet the needs of
applications in which computational resources are limited, and robustness and
interpretability are prioritized. Learning problems can be formulated as
constrained stochastic optimization problems, with the constraints originating
mainly from model assumptions that define a trade-off between complexity and
performance. This trade-off is closely related to over-fitting, generalization
capacity, and robustness to noise and adversarial attacks, and depends on both
the structure and complexity of the model, as well as the properties of the
optimization methods used. We develop an online prototype-based learning
algorithm based on annealing optimization that is formulated as an online
gradient-free stochastic approximation algorithm. The learning model can be
viewed as an interpretable and progressively growing competitive-learning
neural network model to be used for supervised, unsupervised, and reinforcement
learning. The annealing nature of the algorithm contributes to minimal
hyper-parameter tuning requirements, poor local minima prevention, and
robustness with respect to the initial conditions. At the same time, it
provides online control over the performance-complexity trade-off by
progressively increasing the complexity of the learning model as needed,
through an intuitive bifurcation phenomenon. Finally, the use of stochastic
approximation enables the study of the convergence of the learning algorithm
through mathematical tools from dynamical systems and control, and allows for
its integration with reinforcement learning algorithms, constructing an
adaptive state-action aggregation scheme.Comment: arXiv admin note: text overlap with arXiv:2102.0583
Recycle-GAN: Unsupervised Video Retargeting
We introduce a data-driven approach for unsupervised video retargeting that
translates content from one domain to another while preserving the style native
to a domain, i.e., if contents of John Oliver's speech were to be transferred
to Stephen Colbert, then the generated content/speech should be in Stephen
Colbert's style. Our approach combines both spatial and temporal information
along with adversarial losses for content translation and style preservation.
In this work, we first study the advantages of using spatiotemporal constraints
over spatial constraints for effective retargeting. We then demonstrate the
proposed approach for the problems where information in both space and time
matters such as face-to-face translation, flower-to-flower, wind and cloud
synthesis, sunrise and sunset.Comment: ECCV 2018; Please refer to project webpage for videos -
http://www.cs.cmu.edu/~aayushb/Recycle-GA
Task-Driven Dictionary Learning
Modeling data with linear combinations of a few elements from a learned
dictionary has been the focus of much recent research in machine learning,
neuroscience and signal processing. For signals such as natural images that
admit such sparse representations, it is now well established that these models
are well suited to restoration tasks. In this context, learning the dictionary
amounts to solving a large-scale matrix factorization problem, which can be
done efficiently with classical optimization tools. The same approach has also
been used for learning features from data for other purposes, e.g., image
classification, but tuning the dictionary in a supervised way for these tasks
has proven to be more difficult. In this paper, we present a general
formulation for supervised dictionary learning adapted to a wide variety of
tasks, and present an efficient algorithm for solving the corresponding
optimization problem. Experiments on handwritten digit classification, digital
art identification, nonlinear inverse image problems, and compressed sensing
demonstrate that our approach is effective in large-scale settings, and is well
suited to supervised and semi-supervised classification, as well as regression
tasks for data that admit sparse representations.Comment: final draft post-refereein
- …