Structured Learning via Logistic Regression

A successful approach to structured learning is to write the learning objective as a joint function of linear parameters and inference messages, and iterate between updates to each. This paper observes that if the inference problem is "smoothed" through the addition of entropy terms, for fixed messages, the learning objective reduces to a traditional (non-structured) logistic regression problem with respect to parameters. In these logistic regression problems, each training example has a bias term determined by the current set of messages. Based on this insight, the structured energy function can be extended from linear factors to any function class where an "oracle" exists to minimize a logistic loss.Comment: Advances in Neural Information Processing Systems 201

Maximum Likelihood Learning With Arbitrary Treewidth via Fast-Mixing Parameter Sets

Inference is typically intractable in high-treewidth undirected graphical models, making maximum likelihood learning a challenge. One way to overcome this is to restrict parameters to a tractable set, most typically the set of tree-structured parameters. This paper explores an alternative notion of a tractable set, namely a set of "fast-mixing parameters" where Markov chain Monte Carlo (MCMC) inference can be guaranteed to quickly converge to the stationary distribution. While it is common in practice to approximate the likelihood gradient using samples obtained from MCMC, such procedures lack theoretical guarantees. This paper proves that for any exponential family with bounded sufficient statistics, (not just graphical models) when parameters are constrained to a fast-mixing set, gradient descent with gradients approximated by sampling will approximate the maximum likelihood solution inside the set with high-probability. When unregularized, to find a solution epsilon-accurate in log-likelihood requires a total amount of effort cubic in 1/epsilon, disregarding logarithmic factors. When ridge-regularized, strong convexity allows a solution epsilon-accurate in parameter distance with effort quadratic in 1/epsilon. Both of these provide of a fully-polynomial time randomized approximation scheme.Comment: Advances in Neural Information Processing Systems 201

Projecting Ising Model Parameters for Fast Mixing

Inference in general Ising models is difficult, due to high treewidth making tree-based algorithms intractable. Moreover, when interactions are strong, Gibbs sampling may take exponential time to converge to the stationary distribution. We present an algorithm to project Ising model parameters onto a parameter set that is guaranteed to be fast mixing, under several divergences. We find that Gibbs sampling using the projected parameters is more accurate than with the original parameters when interaction strengths are strong and when limited time is available for sampling.Comment: Advances in Neural Information Processing Systems 201

Finito: A Faster, Permutable Incremental Gradient Method for Big Data Problems

Recent advances in optimization theory have shown that smooth strongly convex finite sums can be minimized faster than by treating them as a black box "batch" problem. In this work we introduce a new method in this class with a theoretical convergence rate four times faster than existing methods, for sums with sufficiently many terms. This method is also amendable to a sampling without replacement scheme that in practice gives further speed-ups. We give empirical results showing state of the art performance

An Analysis of Momentum Flux Budgets and Profiles in a Large-Eddy Model

Momentum fluxes and variances play an important role in the characterization and forecast of weather phenomena, but cannot be measured easily. A subdivision of the flux changes into budget terms by the underlying physical processes, such as buoyancy transport, can assist in understanding their sources and influences. Momentum flux and variance budgets for SAM, the System for Atmospheric Modeling, have been implemented and are compared to existing budgets from other simulations. A tool for the visualization of these quantities from three-dimensional grid data has been developed to show and explain their distribution in conjunction with shallow cumulus and stratocumulus clouds. For the shallow cumulus case, we have found opposite fluxes within clouds and in the cloud halo regions as well as a significant contribution in the environment. In a small region within the cloud layer we have upgradient fluxes

Dwelling in Density: A Study on High Density Residential Architecture

This thesis aims to propose an high density plan for Philadelphia by cataloging existing vacancy levels, define variously dense formal typologies within these vacancies, and strategically implement high-density architectures within the existing urban fabric to accomodate future growth