FreezeOut: Accelerate Training by Progressively Freezing Layers

The early layers of a deep neural net have the fewest parameters, but take up the most computation. In this extended abstract, we propose to only train the hidden layers for a set portion of the training run, freezing them out one-by-one and excluding them from the backward pass. Through experiments on CIFAR, we empirically demonstrate that FreezeOut yields savings of up to 20% wall-clock time during training with 3% loss in accuracy for DenseNets, a 20% speedup without loss of accuracy for ResNets, and no improvement for VGG networks. Our code is publicly available at https://github.com/ajbrock/FreezeOutComment: Extended Abstrac

Self-organizing, two-temperature Ising model describing human segregation

A two-temperature Ising-Schelling model is introduced and studied for describing human segregation. The self-organized Ising model with Glauber kinetics simulated by M\"uller et al. exhibits a phase transition between segregated and mixed phases mimicking the change of tolerance (local temperature) of individuals. The effect of external noise is considered here as a second temperature added to the decision of individuals who consider change of accommodation. A numerical evidence is presented for a discontinuous phase transition of the magnetization.Comment: 5 pages, 4 page

SMASH: One-Shot Model Architecture Search through HyperNetworks

Designing architectures for deep neural networks requires expert knowledge and substantial computation time. We propose a technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture. By comparing the relative validation performance of networks with HyperNet-generated weights, we can effectively search over a wide range of architectures at the cost of a single training run. To facilitate this search, we develop a flexible mechanism based on memory read-writes that allows us to define a wide range of network connectivity patterns, with ResNet, DenseNet, and FractalNet blocks as special cases. We validate our method (SMASH) on CIFAR-10 and CIFAR-100, STL-10, ModelNet10, and Imagenet32x32, achieving competitive performance with similarly-sized hand-designed networks. Our code is available at https://github.com/ajbrock/SMAS

Generative and Discriminative Voxel Modeling with Convolutional Neural Networks

When working with three-dimensional data, choice of representation is key. We explore voxel-based models, and present evidence for the viability of voxellated representations in applications including shape modeling and object classification. Our key contributions are methods for training voxel-based variational autoencoders, a user interface for exploring the latent space learned by the autoencoder, and a deep convolutional neural network architecture for object classification. We address challenges unique to voxel-based representations, and empirically evaluate our models on the ModelNet benchmark, where we demonstrate a 51.5% relative improvement in the state of the art for object classification.Comment: 9 pages, 5 figures, 2 table

Free Form Lensing Implications for the Collision of Dark Matter and Gas in the Frontier Fields Cluster MACSJ0416.1-2403

We present a free form mass reconstruction of the massive lensing cluster MACSJ0416.1-2403 using the latest Hubble Frontier Fields data. Our model independent method finds that the extended lensing pattern is generated by two elongated, closely projected clusters of similar mass. Our lens model identifies new lensed images with which we improve the accuracy of the dark matter distribution. We find that the bimodal mass distribution is nearly coincident with the bimodal X-ray emission, but with the two dark matter peaks lying closer together than the centroids of the X-ray emisison. We show this can be achieved if the collision has occurred close to the plane and such that the cores are deflected around each other. The projected mass profiles of both clusters are well constrained because of the many interior lensed images, leading to surprisingly flat mass profiles of both components in the region 15-100 kpc. We discuss the extent to which this may be generated by tidal forces in our dynamical model which are large during an encounter of this type as the cores "graze" each other. The relative velocity between the two cores is estimated to be about 1200 km/s and mostly along the line of sight so that our model is consistent with the relative redshift difference between the two cD galaxies (dz = 0.04).Comment: 22 pages, 18 figures, 2 table

The clinical effectiveness and cost-effectiveness of inhaler devices used in the routine management of chronic asthma in older children: a systematic review and economic evaluation

Background: This review examines the clinical effectiveness and cost-effectiveness of hand-held inhalers to deliver medication for the routine management of chronic asthma in children aged between 5 and 15 years. Asthma is a common disease of the airways, with a prevalence of treated asthma in 5–15-year-olds of around 12% and an actual prevalence in the community as high as 23%. Treatment for the condition is predominantly by inhalation of medication. There are three main types of inhaler device, pressurised metered dose, breath actuated, and dry powder, with the option of the attachment of a spacer to the first two devices under some prescribed circumstances. Two recent reviews have examined the clinical and cost-effectiveness evidence on inhaler devices, but one was for children aged under 5 years and the comparison in the second was made between pressurised metered dose inhalers and other types only. Objectives: This review examines the clinical effectiveness and cost-effectiveness of manual pressurised metered dose inhalers, breath-actuated metered dose inhalers, and breath-actuated dry powder inhalers, with and without spacers as appropriate, to deliver medication for the routine management of chronic asthma in children aged between 5 and 15 years. Methods: Two previous HTA reviews have compared the effectiveness of inhaler devices, one focusing on asthma in children aged under 5 years and the other on asthma and chronic obstructive airways disease in all age groups. For the current review, a literature search was carried out to identify all evidence relating to the use of inhalers in older children with chronic asthma. A search of in-vitro studies undertaken for one of the previous reviews was also updated. The data sources used were: 15 electronic bibliographic databases; the reference lists of one of the previous HTA reports and other relevant articles; health services research-related internet resources; and all sponsor submissions. Studies were selected according to strict inclusion and exclusion criteria, and relevant information concerning effectiveness and patient compliance and preference was extracted directly on to an extraction/evidence table. Quality assurance was monitored. Economic evaluation was undertaken by reviewing existing cost-effective evidence. Further economic modelling was carried out, and tables constructed to determine device cost-minimisation and incremental quality-adjusted life-year (QALY) thresholds between devices. Results: Number and quality of studies, and direction of evidence: Fourteen randomised controlled studies were identified relating to the clinical effectiveness of inhaler devices for delivering β2-agonists. A further five were on devices delivering corticosteroids and one concerned the delivery of cromoglicate. Overall, there were no differences in clinical efficacy between inhaler devices, but a pressurised metered dose inhaler with a spacer would appear to be more effective than one without. These findings endorse those of a previous HTA review but extend them to other inhaler devices. Seven randomised controlled trials examined the impact on clinical effectiveness of using a nonchlorofluorocarbon (CFC) propellant in place of a CFC propellant in metered dose inhalers, both pressurised and breath activated, although only one study considered the latter type. No differences were found between inhalers containing either propellant. A further 30 studies of varying quality, from 12 randomised controlled trials to non-controlled studies, were identified that concerned the impact of use by, and preference for, inhaler type, and treatment adherence in children. Differences between the studies, and limitations in comparative data between various inhaler device types, make it difficult to draw any firm conclusions from this evidence. Summary of benefits: No obvious benefits for one inhaler device type over another for use in children aged 5–15 years were identified. Costs and cost per quality-adjusted life-year: Two approaches have been taken: cost-minimisation and QALY threshold. In the QALY threshold approach, additional QALYs that each device must produce compared with a cheaper device to achieve an acceptable cost per QALY were calculated. Using the cheapest and most expensive devices for delivering 200 μg of beclometasone per day, assuming no cost offset for any device, and a threshold of £5000, the largest QALY needed was 0.00807. With such a small QALY increase, no intervention can be categorically rejected as not cost-effective. Conclusions: Generalisability of findings: On the available evidence there are no obvious benefits for one inhaler device over another when used by children aged 5–15 years with chronic asthma. However, the evidence, in the majority of cases, was compiled on children with mild to moderate asthma and restricted to a limited number of drugs. Therefore the findings may not be generalisable to those at the more severe end of the spectrum of the disease or to inhaler devices delivering some of the drugs used in the management of asthma. Need for further research: Many of the previous studies are likely to have been underpowered. Further clinical trials with a robust methodology, sufficient power and qualitative components are needed to demonstrate any differences in clinical resource use and patients’ asthma symptoms. Further studies should also include the behavioural aspects of patients towards their medication and its delivery mechanisms. It is acknowledged that sufficient power may prove impractical owing to the large numbers of patients required

Millimeter and hard x ray/gamma ray observations of solar flares during the June 1991 GRO campaign

We have carried out high-spatial-resolution millimeter observations of solar flares using the Berkeley-Illinois-Maryland Array (BIMA). At the present time, BIMA consists of only three elements, which is not adequate for mapping highly variable solar phenomena, but is excellent for studies of the temporal structure of flares at millimeter wavelengths at several different spatial scales. We present BIMA observations made during the Gamma Ray Observatories (GRO)/Solar Max 1991 campaign in Jun. 1991 when solar activity was unusually high. Our observations covered the period 8-9 Jun. 1991; this period overlapped the period 4-15 Jun. when the Compton Telescope made the Sun a target of opportunity because of the high level of solar activity

Optimal Estimates for the Electric Field in Two-Dimensions

The purpose of this paper is to set out optimal gradient estimates for solutions to the isotropic conductivity problem in the presence of adjacent conductivity inclusions as the distance between the inclusions goes to zero and their conductivities degenerate. This difficult question arises in the study of composite media. Frequently in composites, the inclusions are very closely spaced and may even touch. It is quite important from a practical point of view to know whether the electric field (the gradient of the potential) can be arbitrarily large as the inclusions get closer to each other or to the boundary of the background medium. In this paper, we establish both upper and lower bounds on the electric field in the case where two circular conductivity inclusions are very close but not touching. We also obtain such bounds when a circular inclusion is very close to the boundary of a circular domain which contains the inclusion. The novelty of these estimates, which improve and make complete our earlier results published in Math. Ann., is that they give an optimal information about the blow-up of the electric field as the conductivities of the inclusions degenerate.Comment: 26 page

Reconfigurable self-sufficient traps for ultracold atoms based on a superconducting square

We report on the trapping of ultracold atoms in the magnetic field formed entirely by persistent supercurrents induced in a thin film type-II superconducting square. The supercurrents are carried by vortices induced in the 2D structure by applying two magnetic field pulses of varying amplitude perpendicular to its surface. This results in a self-sufficient quadrupole trap that does not require any externally applied fields. We investigate the trapping parameters for different supercurrent distributions. Furthermore, to demonstrate possible applications of these types of supercurrent traps we show how a central quadrupole trap can be split into four traps by the use of a bias field.Comment: 5 pages, 7 figure