On the Equivalence between Herding and Conditional Gradient Algorithms

We show that the herding procedure of Welling (2009) takes exactly the form of a standard convex optimization algorithm--namely a conditional gradient algorithm minimizing a quadratic moment discrepancy. This link enables us to invoke convergence results from convex optimization and to consider faster alternatives for the task of approximating integrals in a reproducing kernel Hilbert space. We study the behavior of the different variants through numerical simulations. The experiments indicate that while we can improve over herding on the task of approximating integrals, the original herding algorithm tends to approach more often the maximum entropy distribution, shedding more light on the learning bias behind herding

An optimal finite-dimensional modeling in heat conduction and diffusion equations with partially known eigenstructure

An optimal finite-dimensional modeling technique is presented for a standard class of distributed parameter systems for heat and diffusion equations. A finite-dimensional nominal model with minimum error bounds in frequency domain is established for spectral systems with partially known eigenvalues and eigenfunctions. The result is derived from a completely characterized geometric figure upon complex plane, of all the frequency responses of the systems that have (i) a finite number of given time constants T/sub i/'s and modal coefficients k/sub i/'s, (ii) an upper bound /spl rho/ to the infinite sum of the absolute values of all the modal coefficients k/sub i/'s, (iii) an upper bound T to the unknown T/sub i/'s, and (iv) a given dc gain G(0). Discussions are made on how each parameter mentioned above makes contribution to bounding error or uncertainty, and we stress that steady state analysis for dc input is used effectively in reduced order modeling and bounding errors. The feasibility of the presented scheme is demonstrated by a simple example of heat conduction in ideal copper rod. </p

Seven views on approximate convexity and the geometry of K-spaces

As in Hokusai's series of paintings "Thirty six views of mount Fuji" in which mount Fuji's is sometimes scarcely visible, the central topic of this paper is the geometry of $K$-spaces although in some of the seven views presented $K$-spaces are not easily visible. We study the interplay between the behaviour of approximately convex (and approximately affine) functions on the unit ball of a Banach space and the geometry of Banach K-spaces.Comment: 2 figure

Matrix product states represent ground states faithfully

We quantify how well matrix product states approximate exact ground states of 1-D quantum spin systems as a function of the number of spins and the entropy of blocks of spins. We also investigate the convex set of local reduced density operators of translational invariant systems. The results give a theoretical justification for the high accuracy of renormalization group algorithms, and justifies their use even in the case of critical systems

Use of approximations of Hamilton-Jacobi-Bellman inequality for solving periodic optimization problems

We show that necessary and sufficient conditions of optimality in periodic optimization problems can be stated in terms of a solution of the corresponding HJB inequality, the latter being equivalent to a max-min type variational problem considered on the space of continuously differentiable functions. We approximate the latter with a maximin problem on a finite dimensional subspace of the space of continuously differentiable functions and show that a solution of this problem (existing under natural controllability conditions) can be used for construction of near optimal controls. We illustrate the construction with a numerical example.Comment: 29 pages, 2 figure

A probabilistic interpretation of set-membership filtering: application to polynomial systems through polytopic bounding

Set-membership estimation is usually formulated in the context of set-valued calculus and no probabilistic calculations are necessary. In this paper, we show that set-membership estimation can be equivalently formulated in the probabilistic setting by employing sets of probability measures. Inference in set-membership estimation is thus carried out by computing expectations with respect to the updated set of probability measures P as in the probabilistic case. In particular, it is shown that inference can be performed by solving a particular semi-infinite linear programming problem, which is a special case of the truncated moment problem in which only the zero-th order moment is known (i.e., the support). By writing the dual of the above semi-infinite linear programming problem, it is shown that, if the nonlinearities in the measurement and process equations are polynomial and if the bounding sets for initial state, process and measurement noises are described by polynomial inequalities, then an approximation of this semi-infinite linear programming problem can efficiently be obtained by using the theory of sum-of-squares polynomial optimization. We then derive a smart greedy procedure to compute a polytopic outer-approximation of the true membership-set, by computing the minimum-volume polytope that outer-bounds the set that includes all the means computed with respect to P

Approximations of countably-infinite linear programs over bounded measure spaces

We study a class of countably-infinite-dimensional linear programs (CILPs) whose feasible sets are bounded subsets of appropriately defined weighted spaces of measures. We show how to approximate the optimal value, optimal points, and minimal points of these CILPs by solving finite-dimensional linear programs. The errors of our approximations converge to zero as the size of the finite-dimensional program approaches that of the original problem and are easy to bound in practice. We discuss the use of our methods in the computation of the stationary distributions, occupation measures, and exit distributions of Markov~chains