Lasso and elastic nets by orthants

We propose a new method for computing the lasso path, using the fact that the Manhattan norm of the coefficient vector is linear over every orthant of the parameter space. We use simple calculus and present an algorithm in which the lasso path is series of orthant moves. Our proposal gives the same results as standard literature, with the advantage of neat interpretation of results and explicit lasso formul{\ae}. We extend this proposal to elastic nets and obtain explicit, exact formul{\ae} for the elastic net path, and with a simple change, our lasso algorithm can be used for elastic nets. We present computational examples and provide simple R prototype code.Comment: 44 pages, 10 figures, 3 table

Universal Gröbner Bases for Designs of Experiments

Universal Gröbner bases (UGB) are a useful tool to obtain a set of different models identified by an experimental design. Usually, the algorithms to obtain a UGB for the ideal of a design are computationally intensive. Babson et al. (2003) propose a methodology to construct UGB in polynomial time. Their methodology constructs a list of term orders based upon the Hilbert zonotope. We focus on the generation of such a list. We use results on hyperplane arrangements to present a theorem which simplifies the computation of term orders for designs in two dimensions. Our theorem constructs directly the normal fan of the Hilbert zonotope

Sparse polynomial prediction

In numerical analysis, sparse grids are point configurations used in stochastic finite element approximation, numerical integration and interpolation. This paper is concerned with the construction of polynomial interpolator models in sparse grids. Our proposal stems from the fact that a sparse grid is an echelon design with a hierarchical structure that identifies a single model. We then formulate the model and show that it can be written using inclusion–exclusion formulæ. At this point, we deploy efficient methodologies from the algebraic literature that can simplify considerably the computations. The methodology uses Betti numbers to reduce the number of terms in the inclusion–exclusion while achieving the same result as with exhaustive formulæ

Lasso for hierarchical polynomial models

In a polynomial regression model, the divisibility conditions implicit in polynomial hierarchy give way to a natural construction of constraints for the model parameters. We use this principle to derive versions of strong and weak hierarchy and to extend existing work in the literature, which at the moment is only concerned with models of degree two. We discuss how to estimate parameters in lasso using standard quadratic programming techniques and apply our proposal to both simulated data and examples from the literature. The proposed methodology compares favorably with existing techniques in terms of low validation error and model size

Minimal average degree aberration and the state polytope for experimental designs

For a particular experimental design, there is interest in finding which polynomial models can be identified in the usual regression set up. The algebraic methods based on Groebner bases provide a systematic way of doing this. The algebraic method does not in general produce all estimable models but it can be shown that it yields models which have minimal average degree in a well-defined sense and in both a weighted and unweighted version. This provides an alternative measure to that based on "aberration" and moreover is applicable to any experimental design. A simple algorithm is given and bounds are derived for the criteria, which may be used to give asymptotic Nyquist-like estimability rates as model and sample sizes increase

Generalised design: interpolation and statistical modelling over varieties

In the classical formulation an experimental design is a set of sites at each of which an observation is taken on a response Y . The algebraic method treats the design as giving an "ideal of points" from which potential monomial bases for a polynomial regression can be derived. If the Gröbner basis method is use then the monomial basis depends on the monomial term ordering. The full basis has the same number of terms as the number of design points and gives an exact interpolator for the Y -values over the design points. Here the notation of design point is generalized to a variety. Observation means, in theory, that one observes the value of the response on the variety. A design is a union of varieties and the assumption is, then, that on each variety we observe the response. The task is to construct an interpolator for the function between the varieties. Motivation is provided by transect sampling in a number of fields. Much of the algebraic theory extends to the general case. But special issues arise including the consistency of interpolation at the intersection of the varieties and the consequences of taking a design of points restricted to the varieties

Nonlinear Matroid Optimization and Experimental Design

We study the problem of optimizing nonlinear objective functions over matroids presented by oracles or explicitly. Such functions can be interpreted as the balancing of multi-criteria optimization. We provide a combinatorial polynomial time algorithm for arbitrary oracle-presented matroids, that makes repeated use of matroid intersection, and an algebraic algorithm for vectorial matroids. Our work is partly motivated by applications to minimum-aberration model-fitting in experimental design in statistics, which we discuss and demonstrate in detail

Nonlinear matroid optimization and experimental design

We study the problem of optimizing nonlinear objective functions over matroids presented by oracles or explicitly. Such functions can be interpreted as the balancing of multi-criteria optimization. We provide a combinatorial polynomial time algorithm for arbitrary oracle-presented matroids, that makes repeated use of matroid intersection, and an algebraic algorithm for vectorial matroids. Our work is partly motivated by applications to minimum-aberration model-fitting in experimental design in statistics, which we discuss and demonstrate in detail

Optimal design of measurements on queueing systems

We examine the optimal design of measurements on queues with particular reference to the M/M/1 queue. Using the statistical theory of design of experiments, we calculate numerically the Fisher information matrix for an estimator of the arrival rate and the service rate to find optimal times to measure the queue when the number of measurements are limited for both interfering and non-interfering measurements. We prove that in the non-interfering case, the optimal design is equally spaced. For the interfering case, optimal designs are not necessarily equally spaced. We compute optimal designs for a variety of queuing situations and give results obtained under the $D-$- and $D_s$-optimality criteria