1,703 research outputs found
Optimization with Sparsity-Inducing Penalties
Sparse estimation methods are aimed at using or obtaining parsimonious
representations of data or models. They were first dedicated to linear variable
selection but numerous extensions have now emerged such as structured sparsity
or kernel selection. It turns out that many of the related estimation problems
can be cast as convex optimization problems by regularizing the empirical risk
with appropriate non-smooth norms. The goal of this paper is to present from a
general perspective optimization tools and techniques dedicated to such
sparsity-inducing penalties. We cover proximal methods, block-coordinate
descent, reweighted -penalized techniques, working-set and homotopy
methods, as well as non-convex formulations and extensions, and provide an
extensive set of experiments to compare various algorithms from a computational
point of view
Recommended from our members
FROM OPTIMIZATION TO EQUILIBRATION: UNDERSTANDING AN EMERGING PARADIGM IN ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING
Many existing machine learning (ML) algorithms cannot be viewed as gradient descent on some single objective. The solution trajectories taken by these algorithms naturally exhibit rotation, sometimes forming cycles, a behavior that is not expected with (full-batch) gradient descent. However, these algorithms can be viewed more generally as solving for the equilibrium of a game with possibly multiple competing objectives. Moreover, some recent ML models, specifically generative adversarial networks (GANs) and its variants, are now explicitly formulated as equilibrium problems. Equilibrium problems present challenges beyond those encountered in optimization such as limit-cycles and chaotic attractors and are able to abstract away some of the difficulties encountered when training models like GANs.
In this thesis, I aim to advance our understanding of equilibrium problems so as to improve state-of-the-art in GANs and related domains. In the following chapters, I will present work on designing a no-regret framework for solving monotone equilibrium problems in online or streaming settings (with applications to Reinforcement Learning), ensuring convergence when training a GAN to fit a normal distribution to data by Crossing-the-Curl, improving state-of-the-art image generation with techniques derived from theory, and borrowing tools from dynamical systems theory for analyzing the complex dynamics of GAN training
International Conference on Continuous Optimization (ICCOPT) 2019 Conference Book
The Sixth International Conference on Continuous Optimization took place on the campus of the Technical University of Berlin, August 3-8, 2019. The ICCOPT is a flagship conference of the Mathematical Optimization Society (MOS), organized every three years. ICCOPT 2019 was hosted by the Weierstrass Institute for Applied Analysis and Stochastics (WIAS) Berlin. It included a Summer School and a Conference with a series of plenary and semi-plenary talks, organized and contributed sessions, and poster sessions.
This book comprises the full conference program. It contains, in particular, the scientific program in survey style as well as with all details, and information on the social program, the venue, special meetings, and more
Modification of Gesture-Determined-Dynamic Function with Consideration of Margins for Motion Planning of Humanoid Robots
The gesture-determined-dynamic function (GDDF) offers an effective way to
handle the control problems of humanoid robots. Specifically, GDDF is utilized
to constrain the movements of dual arms of humanoid robots and steer specific
gestures to conduct demanding tasks under certain conditions. However, there is
still a deficiency in this scheme. Through experiments, we found that the
joints of the dual arms, which can be regarded as the redundant manipulators,
could exceed their limits slightly at the joint angle level. The performance
straightly depends on the parameters designed beforehand for the GDDF, which
causes a lack of adaptability to the practical applications of this method. In
this paper, a modified scheme of GDDF with consideration of margins (MGDDF) is
proposed. This MGDDF scheme is based on quadratic programming (QP) framework,
which is widely applied to solving the redundancy resolution problems of robot
arms. Moreover, three margins are introduced in the proposed MGDDF scheme to
avoid joint limits. With consideration of these margins, the joints of
manipulators of the humanoid robots will not exceed their limits, and the
potential damages which might be caused by exceeding limits will be completely
avoided. Computer simulations conducted on MATLAB further verify the
feasibility and superiority of the proposed MGDDF scheme
- …