3,255 research outputs found
SVAG: Stochastic Variance Adjusted Gradient Descent and Biased Stochastic Gradients
We examine biased gradient updates in variance reduced stochastic gradient
methods. For this purpose we introduce SVAG, a SAG/SAGA-like method with
adjustable bias. SVAG is analyzed under smoothness assumptions and we provide
step-size conditions for convergence that match or improve on previously known
conditions for SAG and SAGA. The analysis highlights a step-size requirement
difference between when SVAG is applied to cocoercive operators and when
applied to gradients of smooth functions, a difference not present in ordinary
gradient descent. This difference is verified with numerical experiments. A
variant of SVAG that adaptively selects the bias is presented and compared
numerically to SVAG on a set of classification problems. The adaptive SVAG
frequently performs among the best and always improves on the worst-case
performance of the non-adaptive variant
On distinguishing trees by their chromatic symmetric functions
Let be an unrooted tree. The \emph{chromatic symmetric function} ,
introduced by Stanley, is a sum of monomial symmetric functions corresponding
to proper colorings of . The \emph{subtree polynomial} , first
considered under a different name by Chaudhary and Gordon, is the bivariate
generating function for subtrees of by their numbers of edges and leaves.
We prove that , where is the Hall inner
product on symmetric functions and is a certain symmetric function that
does not depend on . Thus the chromatic symmetric function is a stronger
isomorphism invariant than the subtree polynomial. As a corollary, the path and
degree sequences of a tree can be obtained from its chromatic symmetric
function. As another application, we exhibit two infinite families of trees
(\emph{spiders} and some \emph{caterpillars}), and one family of unicyclic
graphs (\emph{squids}) whose members are determined completely by their
chromatic symmetric functions.Comment: 16 pages, 3 figures. Added references [2], [13], and [15
Fixed Point Iterations for Finite Sum Monotone Inclusions
This thesis studies two families of methods for finding zeros of finite sums of monotone operators, the first being variance-reduced stochastic gradient (VRSG) methods. This is a large family of algorithms that use random sampling to improve the convergence rate compared to more traditional approaches. We examine the optimal sampling distributions and their interaction with the epoch length. Specifically, we show that in methods like SAGA, where the epoch length is directly tied to the random sampling, the optimal sampling becomes more complex compared to for instance L-SVRG, where the epoch length can be chosen independently. We also show that biased VRSG estimates in the style of SAG are sensitive to the problem setting. More precisely, a significantly larger step-size can be used when the monotone operators are cocoercive gradients compared to when they just are cocoercive. This is noteworthy since the standard gradient descent is not affected by this change and the fact that the sensitivity to the problem assumption vanishes when the estimates are unbiased. The second set of methods we examine are deterministic operator splitting methods and we focus on frameworks for constructing and analyzing such splitting methods. One such framework is based on what we call nonlinear resolvents and we present a novel way of ensuring convergence of iterations of nonlinear resolvents by the means of a momentum term. This approach leads in many cases to cheaper per-iteration cost compared to a previously established projection approach. The framework covers many existing methods and we provide a new primal-dual method that uses an extra resolvent step as well as a general approach for adding momentum to any special case of our nonlinear resolvent method. We use a similar concept to the nonlinear resolvent to derive a representation of the entire class of frugal splitting operators, which are splitting operators that use exactly one direct or resolvent evaluation of each operator of the monotone inclusion problem. The representation reveals several new results regarding lifting numbers, existence of solution maps, and parallelizability of the forward/backward evaluations. We show that the minimal lifting is n â 1 â f where n is the number of monotone operators and f is the number of direct evaluations in the splitting. A new convergent and parallelizable frugal splitting operator with minimal lifting is also presented
Optimal Control under Quantised Measurements - A Particle Filter and Reduced Horizon Approach
This thesis covers the optimal control of stochastic systems with coarsely quantised measurements. A particle filter approach is used both for the estimation and control problem. Three main families of particle filters are examined for state estimation, standard SIR filters, SIR filters with generalised sampling and auxiliary filters. A couple of different proposal distributions and weight functions were examined for the generalised SIR and auxiliary filter respectively. The choice of proposal distribution had the greatest impact on performance but the unrivalled best filter was achieved with a combination of generalised sampling and the auxiliary particle filter. For the problem of control the particle filter was used for cost-to-go evaluation by forward simulation in time. Simplifications of the full dynamic programming problem were done by reducing the time horizon resulting in M-measurement feedback policies and a new M-measurement cost feedback policy. One-measurement feedback and M-measurement cost feedback was examined for M 4 and although probing behaviour was observed none of the examined controllers managed to outperform a certainty equivalent controller
Sampling and Update Frequencies in Proximal Variance Reduced Stochastic Gradient Methods
Variance reduced stochastic gradient methods have gained popularity in recent
times. Several variants exist with different strategies for the storing and
sampling of gradients. In this work we focus on the analysis of the interaction
of these two aspects. We present and analyze a general proximal variance
reduced gradient method under strong convexity assumptions. Special cases of
the algorithm include SAGA, L-SVRG and their proximal variants. Our analysis
sheds light on epoch-length selection and the need to balance the convergence
of the iterates and how often gradients are stored. The analysis improves on
other convergence rates found in literature and produces a new and faster
converging sampling strategy for SAGA. Problem instances for which the
predicted rates are the same as the practical rates are presented together with
problems based on real world data.Comment: Fixed unicode-character problems in bibliograph
Identifying ILI Cases from Chief Complaints: Comparing the Accuracy of Keyword and Support Vector Machine Methods
We compared the accuracy of two methods of identifying ILI cases from chief complaints. We found that a support vector machine method was more accurate than a keyword method
Pharmacometric covariate modeling using symbolic regression networks
A central challenge within pharmacometrics is to establish a relation between pharmacological model parameters, such as compartment volumes and diffusion rate constants, and known population covariates, such as age and body mass. There is rich literature dedicated to the learning of functional mappings from the covariates to the model parameters, once a search class of functions has been determined. However, the state-of-the-art selection of the search class itself is ad hoc. We demonstrate how neural network-based symbolic regression can be used to simultaneously find the function form and its parameters. The method is put in relation to the literature on symbolic regression and equation learning. A conceptual demonstration is provided through examples, as is a road map to full-scale employment to pharmacological data sets, relevant to closed-loop anesthesia
Boreal old-growth forest structural diversity challenges aerial photographic survey accuracy
The erosion of old-growth forests in boreal managed landscapes is a major issue currently faced by forest managers; however, resolving this problem requires accurate surveys. The intention of our study was to determine if historic operational aerial forest surveys accurately identified boreal old-growth forests in Quebec, Canada. We first compared stand successional stages (even-aged vs. old-growth) in two aerial surveys performed in 1968 (preindustrial aerial survey) and 2007 (modern aerial survey) on the same 2200 km2 territory. Second, we evaluated the accuracy of the modern aerial survey by comparing its results with those of 74 field plots sampled in the study territory between 2014 and 2016. The two aerial surveys differed significantly; 80.8% of the undisturbed stands that were identified as âold-growthâ in the preindustrial survey were classified as âeven-agedâ in the modern survey, and 60% of the stands identified as âold-growthâ by field sampling were also erroneously identified as âeven-agedâ by the modern aerial survey. The scarcity of obvious old-growth attributes in boreal old-growth forests, as well as poorly adapted modern aerial survey criteria (i.e., criteria requiring high vertical stratification and significant changes in tree species composition along forest succession), were the main factors explaining these errors. It is therefore likely that most of Quebecâs boreal old-growth forests are currently not recognized as such in forest inventories, challenging the efficacy of sustainable forest management policies.
LâĂ©rosion des superficies des vieilles forĂȘts borĂ©ales est actuellement un enjeux majeurs pour les gestionnaires forestier. RĂ©pondre efficacemment Ă cette problĂ©matique demande nĂ©anmoins lâaccĂšs Ă des donnĂ©es dâinventaires fiables. Ainsi, lâobjectif de cette Ă©tude Ă©tait de dĂ©terminer si les inventaires forestiers aĂ©riens identifient correctement les vieilles forĂȘts dans les paysages borĂ©aux du QuĂ©bec, Canada. Nous avons comparĂ© les stades de succession (forĂȘt Ă©quienne ou vieille forĂȘt) de deux inventaires aĂ©riens rĂ©alisĂ©s en 1968 (inventaire aĂ©rien prĂ©industriel) et en 2007 (inventaire aĂ©rien moderne) sur un territoire de 2200 km2. Nous avons aussi comparĂ© les rĂ©sultats de lâinventaire aĂ©rien moderne avec ceux obtenus Ă partir de 74 placettes de terrain Ă©chantillonnĂ©s entre 2014 et 2016. Les deux inventaires aĂ©riens Ă©taient trĂšs incohĂ©rents : 80.8% des peuplements non-perturbĂ©s identifiĂ©s comme « vieilles forĂȘts » par lâinventaire prĂ©industriel Ă©taient classĂ©s comme « Ă©quiens » par lâinventaire moderne. 60% des placettes de terrain identifiĂ©es comme vieilles forĂȘts Ă©taient aussi classĂ©es « Ă©quiens » par lâinventaire aĂ©rien moderne. Le manque dâattributs de vieilles forĂȘts Ă©vidents ainsi que lâutilisation de critĂšres inadaptĂ©s (câest-Ă -dire nĂ©cessitant une forte complexitĂ© verticale et dâimportants changements de composition en espĂšces arborescentes durant la succession forestiĂšre) Ă©taient les principaux Ă©lĂ©ments expliquant ces erreurs. Il est ainsi possible que la majoritĂ© des vieilles forĂȘts borĂ©ales du QuĂ©bec ne soient pas identifiĂ©s comme telles, limitant lâefficacitĂ© des stratĂ©gies de gestion durable
- âŠ