FreezeOut: Accelerate Training by Progressively Freezing Layers

The early layers of a deep neural net have the fewest parameters, but take up the most computation. In this extended abstract, we propose to only train the hidden layers for a set portion of the training run, freezing them out one-by-one and excluding them from the backward pass. Through experiments on CIFAR, we empirically demonstrate that FreezeOut yields savings of up to 20% wall-clock time during training with 3% loss in accuracy for DenseNets, a 20% speedup without loss of accuracy for ResNets, and no improvement for VGG networks. Our code is publicly available at https://github.com/ajbrock/FreezeOutComment: Extended Abstrac

SMASH: One-Shot Model Architecture Search through HyperNetworks

Designing architectures for deep neural networks requires expert knowledge and substantial computation time. We propose a technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture. By comparing the relative validation performance of networks with HyperNet-generated weights, we can effectively search over a wide range of architectures at the cost of a single training run. To facilitate this search, we develop a flexible mechanism based on memory read-writes that allows us to define a wide range of network connectivity patterns, with ResNet, DenseNet, and FractalNet blocks as special cases. We validate our method (SMASH) on CIFAR-10 and CIFAR-100, STL-10, ModelNet10, and Imagenet32x32, achieving competitive performance with similarly-sized hand-designed networks. Our code is available at https://github.com/ajbrock/SMAS

Generative and Discriminative Voxel Modeling with Convolutional Neural Networks

When working with three-dimensional data, choice of representation is key. We explore voxel-based models, and present evidence for the viability of voxellated representations in applications including shape modeling and object classification. Our key contributions are methods for training voxel-based variational autoencoders, a user interface for exploring the latent space learned by the autoencoder, and a deep convolutional neural network architecture for object classification. We address challenges unique to voxel-based representations, and empirically evaluate our models on the ModelNet benchmark, where we demonstrate a 51.5% relative improvement in the state of the art for object classification.Comment: 9 pages, 5 figures, 2 table

Time Dependent Clustering Analysis of the Second BATSE Gamma-Ray Burst Catalog

A time dependent two-point correlation-function analysis of the BATSE 2B catalog finds no evidence of burst repetition. As part of this analysis, we discuss the effects of sky exposure on the observability of burst repetition and present the equation describing the signature of burst repetition in the data. For a model of all burst repetition from a source occurring in less than five days we derive upper limits on the number of bursts in the catalog from repeaters and model-dependent upper limits on the fraction of burst sources that produce multiple outbursts.Comment: To appear in the Astrophysical Journal Letters, uuencoded compressed PostScript, 11 pages with 4 embedded figure

Warped Supersymmetric Unification with Non-Unified Superparticle Spectrum

We present a new supersymmetric extension of the standard model. The model is constructed in warped space, with a unified bulk symmetry broken by boundary conditions on both the Planck and TeV branes. In the supersymmetric limit, the massless spectrum contains exotic colored particles along with the particle content of the minimal supersymmetric standard model (MSSM). Nevertheless, the model still reproduces the MSSM prediction for gauge coupling unification and does not suffer from a proton decay problem. The exotic states acquire masses from supersymmetry breaking, making the model completely viable, but there is still the possibility that these states will be detected at the LHC. The lightest of these states is most likely A_5^XY, the fifth component of the gauge field associated with the broken unified symmetry. Because supersymmetry is broken on the SU(5)-violating TeV brane, the gaugino masses generated at the TeV scale are completely independent of one another. We explore some of the unusual features that the superparticle spectrum might have as a consequence.Comment: 21 pages, Latex, version to appear in Phys. Rev.

Maori octopus (Octopus maorum) bycatch and southern rock lobster (Jasus edwardsii) mortality in the South Australian lobster fishery

Octopuses are commonly taken as bycatch in many trap fisheries for spiny lobsters (Decapoda: Palinuridae) and can cause significant levels of within-trap lobster mortality. This article describes spatiotemporal patterns for Maori octopus (Octopus maorum) catch rates and rock lobster (Jasus edwardsii) mortality rates and examines factors that are associated with within-trap lobster mortality in the South Australian rock lobster fishery (SARLF). Since 1983, between 38,000 and 119,000 octopuses per annum have been taken in SARLF traps. Catch rates have fluctuated between 2.2 and 6.2 octopus/100 trap-lifts each day. There is no evidence to suggest that catch rates have declined or that this level of bycatch is unsustainable. Over the last five years, approximately 240,000 lobsters per annum have been killed in traps, representing ~4% of the total catch. Field studies show that over 98% of within-trap lobster mortality is attributable to octopus predation. Lobster mortality rates are positively correlated with the catch rates of octopus. The highest octopus catch rates and lobster mortality rates are recorded during summer and in the more productive southern zone of the fishery. In the southern zone, within-trap lobster mortality rates have increased in recent years, apparently in response to the increase in the number of lobsters in traps and the resultant increase in the probability of octopus encountering traps containing one or more lobsters. Lobster mortality rates are also positively correlated with soak-times in the southern zone fishery and with lobster size. Minimizing trap soak-times is one method currently available for reducing lobster mortality rates. More significant reductions in the rates of within-trap lobster mortality may require a change in the design of lobster traps

Raising, Spending, and Regulating Party Finances in the Provinces

Money is essential for political parties and candidates. In this article, we probe the complicated relationship between money and electoral politics. Using the cases of Manitoba, Saskatchewan and British Columbia, we explore the potential for provincial political systems to serve as laboratories to examine how different regulatory regimes affect political financing. As such, this article is an exploratory analysis of the potential for the comparative study of provincial campaign finance

Transport in Transitory, Three-Dimensional, Liouville Flows

We derive an action-flux formula to compute the volumes of lobes quantifying transport between past- and future-invariant Lagrangian coherent structures of n-dimensional, transitory, globally Liouville flows. A transitory system is one that is nonautonomous only on a compact time interval. This method requires relatively little Lagrangian information about the codimension-one surfaces bounding the lobes, relying only on the generalized actions of loops on the lobe boundaries. These are easily computed since the vector fields are autonomous before and after the time-dependent transition. Two examples in three-dimensions are studied: a transitory ABC flow and a model of a microdroplet moving through a microfluidic channel mixer. In both cases the action-flux computations of transport are compared to those obtained using Monte Carlo methods.Comment: 30 pages, 16 figures, 1 table, submitted to SIAM J. Appl. Dyn. Sy