Automatic Differentiation Tools in Optimization Software

We discuss the role of automatic differentiation tools in optimization software. We emphasize issues that are important to large-scale optimization and that have proved useful in the installation of nonlinear solvers in the NEOS Server. Our discussion centers on the computation of the gradient and Hessian matrix for partially separable functions and shows that the gradient and Hessian matrix can be computed with guaranteed bounds in time and memory requirementsComment: 11 page

Comparing estimation methods for spatial econometrics techniques using R.

Recent advances in spatial econometrics model fitting techniques have made it more desirable to be able to compare results and timings. Results should correspond between implementations using different applications, while timings are more readily compared within a single application. A broad range of model fitting techniques are provided by the contributed R packages for spatial econometrics. These model fitting techniques are associated with methods for estimating impacts and some tests, which will also be presented and compared. This review constitutes an up-to-date demonstration of techniques now available in R, and mentions some that will shortly become more generally available.Spatial autoregression; Econometric software.

Persistent Homology in Sparse Regression and its Application to Brain Morphometry

Sparse systems are usually parameterized by a tuning parameter that determines the sparsity of the system. How to choose the right tuning parameter is a fundamental and difficult problem in learning the sparse system. In this paper, by treating the the tuning parameter as an additional dimension, persistent homological structures over the parameter space is introduced and explored. The structures are then further exploited in speeding up the computation using the proposed soft-thresholding technique. The topological structures are further used as multivariate features in the tensor-based morphometry (TBM) in characterizing white matter alterations in children who have experienced severe early life stress and maltreatment. These analyses reveal that stress-exposed children exhibit more diffuse anatomical organization across the whole white matter region.Comment: submitted to IEEE Transactions on Medical Imagin

Scalable Rejection Sampling for Bayesian Hierarchical Models

Bayesian hierarchical modeling is a popular approach to capturing unobserved heterogeneity across individual units. However, standard estimation methods such as Markov chain Monte Carlo (MCMC) can be impracticable for modeling outcomes from a large number of units. We develop a new method to sample from posterior distributions of Bayesian models, without using MCMC. Samples are independent, so they can be collected in parallel, and we do not need to be concerned with issues like chain convergence and autocorrelation. The algorithm is scalable under the weak assumption that individual units are conditionally independent, making it applicable for large datasets. It can also be used to compute marginal likelihoods

The CPR Method and Beyond : Prologue

In $1974$ A.R. Curtis, M.J.D. Powell, and J.K.Reid published a seminal paper on the estimation of Jacobian matrices which was later coined as the CPR method. Central to the CPR method is the effective utilization of a priori known sparsity information. It is only recently that the optimal CPR method in its general form is characterized and the theoretical underpinning for the optimality is shown. In this short note we provide an overview of the development of computational techniques and software tools for the estimation of Jacobian matrices