SVAG: Stochastic Variance Adjusted Gradient Descent and Biased Stochastic Gradients

We examine biased gradient updates in variance reduced stochastic gradient methods. For this purpose we introduce SVAG, a SAG/SAGA-like method with adjustable bias. SVAG is analyzed under smoothness assumptions and we provide step-size conditions for convergence that match or improve on previously known conditions for SAG and SAGA. The analysis highlights a step-size requirement difference between when SVAG is applied to cocoercive operators and when applied to gradients of smooth functions, a difference not present in ordinary gradient descent. This difference is verified with numerical experiments. A variant of SVAG that adaptively selects the bias is presented and compared numerically to SVAG on a set of classification problems. The adaptive SVAG frequently performs among the best and always improves on the worst-case performance of the non-adaptive variant

On distinguishing trees by their chromatic symmetric functions

Let $T$ be an unrooted tree. The \emph{chromatic symmetric function} $X_T$, introduced by Stanley, is a sum of monomial symmetric functions corresponding to proper colorings of $T$. The \emph{subtree polynomial} $S_T$, first considered under a different name by Chaudhary and Gordon, is the bivariate generating function for subtrees of $T$ by their numbers of edges and leaves. We prove that $S_T =$, where $$ is the Hall inner product on symmetric functions and $\Phi$ is a certain symmetric function that does not depend on $T$. Thus the chromatic symmetric function is a stronger isomorphism invariant than the subtree polynomial. As a corollary, the path and degree sequences of a tree can be obtained from its chromatic symmetric function. As another application, we exhibit two infinite families of trees (\emph{spiders} and some \emph{caterpillars}), and one family of unicyclic graphs (\emph{squids}) whose members are determined completely by their chromatic symmetric functions.Comment: 16 pages, 3 figures. Added references [2], [13], and [15

Fixed Point Iterations for Finite Sum Monotone Inclusions

This thesis studies two families of methods for finding zeros of finite sums of monotone operators, the first being variance-reduced stochastic gradient (VRSG) methods. This is a large family of algorithms that use random sampling to improve the convergence rate compared to more traditional approaches. We examine the optimal sampling distributions and their interaction with the epoch length. Specifically, we show that in methods like SAGA, where the epoch length is directly tied to the random sampling, the optimal sampling becomes more complex compared to for instance L-SVRG, where the epoch length can be chosen independently. We also show that biased VRSG estimates in the style of SAG are sensitive to the problem setting. More precisely, a significantly larger step-size can be used when the monotone operators are cocoercive gradients compared to when they just are cocoercive. This is noteworthy since the standard gradient descent is not affected by this change and the fact that the sensitivity to the problem assumption vanishes when the estimates are unbiased. The second set of methods we examine are deterministic operator splitting methods and we focus on frameworks for constructing and analyzing such splitting methods. One such framework is based on what we call nonlinear resolvents and we present a novel way of ensuring convergence of iterations of nonlinear resolvents by the means of a momentum term. This approach leads in many cases to cheaper per-iteration cost compared to a previously established projection approach. The framework covers many existing methods and we provide a new primal-dual method that uses an extra resolvent step as well as a general approach for adding momentum to any special case of our nonlinear resolvent method. We use a similar concept to the nonlinear resolvent to derive a representation of the entire class of frugal splitting operators, which are splitting operators that use exactly one direct or resolvent evaluation of each operator of the monotone inclusion problem. The representation reveals several new results regarding lifting numbers, existence of solution maps, and parallelizability of the forward/backward evaluations. We show that the minimal lifting is n − 1 − f where n is the number of monotone operators and f is the number of direct evaluations in the splitting. A new convergent and parallelizable frugal splitting operator with minimal lifting is also presented

Optimal Control under Quantised Measurements - A Particle Filter and Reduced Horizon Approach

This thesis covers the optimal control of stochastic systems with coarsely quantised measurements. A particle filter approach is used both for the estimation and control problem. Three main families of particle filters are examined for state estimation, standard SIR filters, SIR filters with generalised sampling and auxiliary filters. A couple of different proposal distributions and weight functions were examined for the generalised SIR and auxiliary filter respectively. The choice of proposal distribution had the greatest impact on performance but the unrivalled best filter was achieved with a combination of generalised sampling and the auxiliary particle filter. For the problem of control the particle filter was used for cost-to-go evaluation by forward simulation in time. Simplifications of the full dynamic programming problem were done by reducing the time horizon resulting in M-measurement feedback policies and a new M-measurement cost feedback policy. One-measurement feedback and M-measurement cost feedback was examined for M 4 and although probing behaviour was observed none of the examined controllers managed to outperform a certainty equivalent controller

Sampling and Update Frequencies in Proximal Variance Reduced Stochastic Gradient Methods

Variance reduced stochastic gradient methods have gained popularity in recent times. Several variants exist with different strategies for the storing and sampling of gradients. In this work we focus on the analysis of the interaction of these two aspects. We present and analyze a general proximal variance reduced gradient method under strong convexity assumptions. Special cases of the algorithm include SAGA, L-SVRG and their proximal variants. Our analysis sheds light on epoch-length selection and the need to balance the convergence of the iterates and how often gradients are stored. The analysis improves on other convergence rates found in literature and produces a new and faster converging sampling strategy for SAGA. Problem instances for which the predicted rates are the same as the practical rates are presented together with problems based on real world data.Comment: Fixed unicode-character problems in bibliograph

Identifying ILI Cases from Chief Complaints: Comparing the Accuracy of Keyword and Support Vector Machine Methods

We compared the accuracy of two methods of identifying ILI cases from chief complaints. We found that a support vector machine method was more accurate than a keyword method

Pharmacometric covariate modeling using symbolic regression networks

A central challenge within pharmacometrics is to establish a relation between pharmacological model parameters, such as compartment volumes and diffusion rate constants, and known population covariates, such as age and body mass. There is rich literature dedicated to the learning of functional mappings from the covariates to the model parameters, once a search class of functions has been determined. However, the state-of-the-art selection of the search class itself is ad hoc. We demonstrate how neural network-based symbolic regression can be used to simultaneously find the function form and its parameters. The method is put in relation to the literature on symbolic regression and equation learning. A conceptual demonstration is provided through examples, as is a road map to full-scale employment to pharmacological data sets, relevant to closed-loop anesthesia

Boreal old-growth forest structural diversity challenges aerial photographic survey accuracy

The erosion of old-growth forests in boreal managed landscapes is a major issue currently faced by forest managers; however, resolving this problem requires accurate surveys. The intention of our study was to determine if historic operational aerial forest surveys accurately identified boreal old-growth forests in Quebec, Canada. We first compared stand successional stages (even-aged vs. old-growth) in two aerial surveys performed in 1968 (preindustrial aerial survey) and 2007 (modern aerial survey) on the same 2200 km2 territory. Second, we evaluated the accuracy of the modern aerial survey by comparing its results with those of 74 field plots sampled in the study territory between 2014 and 2016. The two aerial surveys differed significantly; 80.8% of the undisturbed stands that were identified as “old-growth” in the preindustrial survey were classified as “even-aged” in the modern survey, and 60% of the stands identified as “old-growth” by field sampling were also erroneously identified as “even-aged” by the modern aerial survey. The scarcity of obvious old-growth attributes in boreal old-growth forests, as well as poorly adapted modern aerial survey criteria (i.e., criteria requiring high vertical stratification and significant changes in tree species composition along forest succession), were the main factors explaining these errors. It is therefore likely that most of Quebec’s boreal old-growth forests are currently not recognized as such in forest inventories, challenging the efficacy of sustainable forest management policies. L’érosion des superficies des vieilles forêts boréales est actuellement un enjeux majeurs pour les gestionnaires forestier. Répondre efficacemment à cette problématique demande néanmoins l’accès à des données d’inventaires fiables. Ainsi, l’objectif de cette étude était de déterminer si les inventaires forestiers aériens identifient correctement les vieilles forêts dans les paysages boréaux du Québec, Canada. Nous avons comparé les stades de succession (forêt équienne ou vieille forêt) de deux inventaires aériens réalisés en 1968 (inventaire aérien préindustriel) et en 2007 (inventaire aérien moderne) sur un territoire de 2200 km2. Nous avons aussi comparé les résultats de l’inventaire aérien moderne avec ceux obtenus à partir de 74 placettes de terrain échantillonnés entre 2014 et 2016. Les deux inventaires aériens étaient très incohérents : 80.8% des peuplements non-perturbés identifiés comme « vieilles forêts » par l’inventaire préindustriel étaient classés comme « équiens » par l’inventaire moderne. 60% des placettes de terrain identifiées comme vieilles forêts étaient aussi classées « équiens » par l’inventaire aérien moderne. Le manque d’attributs de vieilles forêts évidents ainsi que l’utilisation de critères inadaptés (c’est-à-dire nécessitant une forte complexité verticale et d’importants changements de composition en espèces arborescentes durant la succession forestière) étaient les principaux éléments expliquant ces erreurs. Il est ainsi possible que la majorité des vieilles forêts boréales du Québec ne soient pas identifiés comme telles, limitant l’efficacité des stratégies de gestion durable