284 research outputs found
Logistic Regression: Tight Bounds for Stochastic and Online Optimization
The logistic loss function is often advocated in machine learning and
statistics as a smooth and strictly convex surrogate for the 0-1 loss. In this
paper we investigate the question of whether these smoothness and convexity
properties make the logistic loss preferable to other widely considered options
such as the hinge loss. We show that in contrast to known asymptotic bounds, as
long as the number of prediction/optimization iterations is sub exponential,
the logistic loss provides no improvement over a generic non-smooth loss
function such as the hinge loss. In particular we show that the convergence
rate of stochastic logistic optimization is bounded from below by a polynomial
in the diameter of the decision set and the number of prediction iterations,
and provide a matching tight upper bound. This resolves the COLT open problem
of McMahan and Streeter (2012)
Beyond Convexity: Stochastic Quasi-Convex Optimization
Stochastic convex optimization is a basic and well studied primitive in
machine learning. It is well known that convex and Lipschitz functions can be
minimized efficiently using Stochastic Gradient Descent (SGD). The Normalized
Gradient Descent (NGD) algorithm, is an adaptation of Gradient Descent, which
updates according to the direction of the gradients, rather than the gradients
themselves. In this paper we analyze a stochastic version of NGD and prove its
convergence to a global minimum for a wider class of functions: we require the
functions to be quasi-convex and locally-Lipschitz. Quasi-convexity broadens
the con- cept of unimodality to multidimensions and allows for certain types of
saddle points, which are a known hurdle for first-order optimization methods
such as gradient descent. Locally-Lipschitz functions are only required to be
Lipschitz in a small region around the optimum. This assumption circumvents
gradient explosion, which is another known hurdle for gradient descent
variants. Interestingly, unlike the vanilla SGD algorithm, the stochastic
normalized gradient descent algorithm provably requires a minimal minibatch
size
On Graduated Optimization for Stochastic Non-Convex Problems
The graduated optimization approach, also known as the continuation method,
is a popular heuristic to solving non-convex problems that has received renewed
interest over the last decade. Despite its popularity, very little is known in
terms of theoretical convergence analysis. In this paper we describe a new
first-order algorithm based on graduated optimiza- tion and analyze its
performance. We characterize a parameterized family of non- convex functions
for which this algorithm provably converges to a global optimum. In particular,
we prove that the algorithm converges to an {\epsilon}-approximate solution
within O(1/\epsilon^2) gradient-based steps. We extend our algorithm and
analysis to the setting of stochastic non-convex optimization with noisy
gradient feedback, attaining the same convergence rate. Additionally, we
discuss the setting of zero-order optimization, and devise a a variant of our
algorithm which converges at rate of O(d^2/\epsilon^4).Comment: 17 page
Quantification of hemodynamic changes induced by virtual placement of multiple stents across a wide -necked Basilar trunk aneurysm
OBJECTIVE: The porous intravascular stents that are currently available may not cause
complete aneurysm thrombosis and may therefore fail to provide durable protection
against aneurysm rupture when used as a sole treatment modality. The goal of this study
was to quantify the effects of porous stents on aneurysm hemodynamics using computational fluid dynamics.
METHODS: The geometry of a wide-necked saccular basilar trunk aneurysm was reconstructed from a patient’s computed tomographic angiography images. Three commercial stents (Neuroform2; Boston Scientific/Target, San Leandro, CA; Wingspan; Boston
Scientific, Fremont, CA; and Vision; Guidant Corp., Santa Clara, CA) were modeled.
Various combinations of one to three stents were virtually conformed to fit into the vessel lumen and placed across the aneurysm orifice. An unstented aneurysm served as a
control. Computational fluid dynamics analysis was performed to calculate the hemodynamic parameters considered important in aneurysm pathogenesis and thrombosis
for each of the models.
RESULTS: The complex flow pattern observed in the unstented aneurysm was suppressed by stenting. Stent placement lowered the wall shear stress in the aneurysm,
and this effect was increased by additional stent deployment. Turnover time was moderately increased after single- and double-stent placement and markedly increased after
three stents were placed. The influence of stent design on hemodynamic parameters
was more significant in double-stented models than in other models.
CONCLUSION: Aneurysm hemodynamic parameters were significantly modified
by placement of multiple stents. Because the associated modifications may be helpful as well as harmful in terms of rupture risk, use of this technique requires careful
consideration
- …