A Family of Subgradient-Based Methods for Convex Optimization Problems in a Unifying Framework

We propose a new family of subgradient- and gradient-based methods which converges with optimal complexity for convex optimization problems whose feasible region is simple enough. This includes cases where the objective function is non-smooth, smooth, have composite/saddle structure, or are given by an inexact oracle model. We unified the way of constructing the subproblems which are necessary to be solved at each iteration of these methods. This permitted us to analyze the convergence of these methods in a unified way compared to previous results which required different approaches for each method/algorithm. Our contribution rely on two well-known methods in non-smooth convex optimization: the mirror-descent method by Nemirovski-Yudin and the dual-averaging method by Nesterov. Therefore, our family of methods includes them and many other methods as particular cases. For instance, the proposed family of classical gradient methods and its accelerations generalize Devolder et al.'s, Nesterov's primal/dual gradient methods, and Tseng's accelerated proximal gradient methods. Also our family of methods can partially become special cases of other universal methods, too. As an additional contribution, the novel extended mirror-descent method removes the compactness assumption of the feasible region and the fixation of the total number of iterations which is required by the original mirror-descent method in order to attain the optimal complexity.Comment: 31 pages. v3: Major revision. Research Report B-477, Department of Mathematical and Computing Sciences, Tokyo Institute of Technology, February 201

The Fate of the First Galaxies. I. Self-Consistent Cosmological Simulations with Radiative Transfer

In cold dark matter (CDM) cosmogonies, low-mass objects play an important role in the evolution of the universe. Not only are they the first luminous objects to shed light in a previously dark universe, but, if their formation is not inhibited by their own feedback, they dominate the galaxy mass function until redshift z \sim 5. In this paper we present and discuss the implementation of a 3D cosmological code that includes most of the needed physics to simulate the formation and evolution of the first galaxies with a self-consistent treatment of radiative feedback. The simulation includes continuum radiative transfer using the ``Optically Thin Variable Eddington Tensor'' (OTVET) approximation and line-radiative transfer in the H_2 Lyman-Werner bands of the background radiation. We include detailed chemistry for H_2 formation/destruction, molecular and atomic cooling/heating processes, ionization by secondary electrons, and heating by Ly\alpha resonant scattering. We find that the first galaxies ("small-halos") are characterized by a bursting star formation, self-regulated by a feedback process that acts on cosmological scales. Their formation is not suppressed by feedback processes; therefore, their impact on cosmic evolution cannot be neglected. The main focus of this paper is on the methodology of the simulations, and we only briefly introduce some of the results. An extensive discussion of the results and the nature of the feedback mechanism are the focus of a companion paper.Comment: Accepted for publication on ApJ, 33 pages, including 14 figures and 2 tables. Movies and a higher quality version of the paper (figures) are available at: http://casa.colorado.edu/~ricotti/MOVIES.htm

The Modified Direct Method: an Approach for Smoothing Planar and Surface Meshes

The Modified Direct Method (MDM) is an iterative mesh smoothing method for smoothing planar and surface meshes, which is developed from the non-iterative smoothing method originated by Balendran [1]. When smooth planar meshes, the performance of the MDM is effectively identical to that of Laplacian smoothing, for triangular and quadrilateral meshes; however, the MDM outperforms Laplacian smoothing for tri-quad meshes. When smooth surface meshes, for trian-gular, quadrilateral and quad-dominant mixed meshes, the mean quality(MQ) of all mesh elements always increases and the mean square error (MSE) decreases during smoothing; For tri-dominant mixed mesh, the quality of triangles always descends while that of quads ascends. Test examples show that the MDM is convergent for both planar and surface triangular, quadrilateral and tri-quad meshes.Comment: 18 page

A Novel Frank-Wolfe Algorithm. Analysis and Applications to Large-Scale SVM Training

Recently, there has been a renewed interest in the machine learning community for variants of a sparse greedy approximation procedure for concave optimization known as {the Frank-Wolfe (FW) method}. In particular, this procedure has been successfully applied to train large-scale instances of non-linear Support Vector Machines (SVMs). Specializing FW to SVM training has allowed to obtain efficient algorithms but also important theoretical results, including convergence analysis of training algorithms and new characterizations of model sparsity. In this paper, we present and analyze a novel variant of the FW method based on a new way to perform away steps, a classic strategy used to accelerate the convergence of the basic FW procedure. Our formulation and analysis is focused on a general concave maximization problem on the simplex. However, the specialization of our algorithm to quadratic forms is strongly related to some classic methods in computational geometry, namely the Gilbert and MDM algorithms. On the theoretical side, we demonstrate that the method matches the guarantees in terms of convergence rate and number of iterations obtained by using classic away steps. In particular, the method enjoys a linear rate of convergence, a result that has been recently proved for MDM on quadratic forms. On the practical side, we provide experiments on several classification datasets, and evaluate the results using statistical tests. Experiments show that our method is faster than the FW method with classic away steps, and works well even in the cases in which classic away steps slow down the algorithm. Furthermore, these improvements are obtained without sacrificing the predictive accuracy of the obtained SVM model.Comment: REVISED VERSION (October 2013) -- Title and abstract have been revised. Section 5 was added. Some proofs have been summarized (full-length proofs available in the previous version

MDFEM: Multivariate decomposition finite element method for elliptic PDEs with lognormal diffusion coefficients using higher-order QMC and FEM

We introduce the multivariate decomposition finite element method for elliptic PDEs with lognormal diffusion coefficient $a=\exp(Z)$ where $Z$ is a Gaussian random field defined by an infinite series expansion $Z(\boldsymbol{y}) = \sum_{j \ge 1} y_j \, \phi_j$ with $y_j \sim \mathcal{N}(0,1)$ and a given sequence of functions $\{\phi_j\}_{j \ge 1}$. We use the MDFEM to approximate the expected value of a linear functional of the solution of the PDE which is an infinite-dimensional integral over the parameter space. The proposed algorithm uses the multivariate decomposition method to compute the infinite-dimensional integral by a decomposition into finite-dimensional integrals, which we resolve using quasi-Monte Carlo methods, and for which we use the finite element method to solve different instances of the PDE. We develop higher-order quasi-Monte Carlo rules for integration over the finite-dimensional Euclidean space with respect to the Gaussian distribution by use of a truncation strategy. By linear transformations of interlaced polynomial lattice rules from the unit cube to a multivariate box of the Euclidean space we achieve higher-order convergence rates for functions belonging to a class of anchored Gaussian Sobolev spaces while taking into account the truncation error. Under appropriate conditions, the MDFEM achieves higher-order convergence rates in term of error versus cost, i.e., to achieve an accuracy of $O(\epsilon)$ the computational cost is $O(\epsilon^{-1/\lambda-d'/\lambda}) = O(\epsilon^{-(p^* + d'/\tau)/(1-p^*)})$ where $\epsilon^{-1/\lambda}$ and $\epsilon^{-d'/\lambda}$ are respectively the cost of the quasi-Monte Carlo cubature and the finite element approximations, with $d' = d \, (1+\delta')$ for some $\delta' \ge 0$ and $d$ the physical dimension, and $0 < p^* \le (2 + d'/\tau)^{-1}$ is a parameter representing the sparsity of $\{\phi_j\}_{j \ge 1}$.Comment: 48 page

Topology Detection in Microgrids with Micro-Synchrophasors

Network topology in distribution networks is often unknown, because most switches are not equipped with measurement devices and communication links. However, knowledge about the actual topology is critical for safe and reliable grid operation. This paper proposes a voting-based topology detection method based on micro-synchrophasor measurements. The minimal difference between measured and calculated voltage angle or voltage magnitude, respectively, indicates the actual topology. Micro-synchrophasors or micro-Phasor Measurement Units ({\mu}PMU) are high-precision devices that can measure voltage angle differences on the order of ten millidegrees. This accuracy is important for distribution networks due to the smaller angle differences as compared to transmission networks. For this paper, a microgrid test bed is implemented in MATLAB with simulated measurements from {\mu}PMUs as well as SCADA measurement devices. The results show that topologies can be detected with high accuracy. Additionally, topology detection by voltage angle shows better results than detection by voltage magnitude.Comment: 5 Pages, PESGM2015, Denver, C