FreezeOut: Accelerate Training by Progressively Freezing Layers

The early layers of a deep neural net have the fewest parameters, but take up the most computation. In this extended abstract, we propose to only train the hidden layers for a set portion of the training run, freezing them out one-by-one and excluding them from the backward pass. Through experiments on CIFAR, we empirically demonstrate that FreezeOut yields savings of up to 20% wall-clock time during training with 3% loss in accuracy for DenseNets, a 20% speedup without loss of accuracy for ResNets, and no improvement for VGG networks. Our code is publicly available at https://github.com/ajbrock/FreezeOutComment: Extended Abstrac

SMASH: One-Shot Model Architecture Search through HyperNetworks

Designing architectures for deep neural networks requires expert knowledge and substantial computation time. We propose a technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture. By comparing the relative validation performance of networks with HyperNet-generated weights, we can effectively search over a wide range of architectures at the cost of a single training run. To facilitate this search, we develop a flexible mechanism based on memory read-writes that allows us to define a wide range of network connectivity patterns, with ResNet, DenseNet, and FractalNet blocks as special cases. We validate our method (SMASH) on CIFAR-10 and CIFAR-100, STL-10, ModelNet10, and Imagenet32x32, achieving competitive performance with similarly-sized hand-designed networks. Our code is available at https://github.com/ajbrock/SMAS

Generative and Discriminative Voxel Modeling with Convolutional Neural Networks

When working with three-dimensional data, choice of representation is key. We explore voxel-based models, and present evidence for the viability of voxellated representations in applications including shape modeling and object classification. Our key contributions are methods for training voxel-based variational autoencoders, a user interface for exploring the latent space learned by the autoencoder, and a deep convolutional neural network architecture for object classification. We address challenges unique to voxel-based representations, and empirically evaluate our models on the ModelNet benchmark, where we demonstrate a 51.5% relative improvement in the state of the art for object classification.Comment: 9 pages, 5 figures, 2 table

Multiple origins of extra electron diffractions in fcc metals

Diffuse intensities in the electron diffraction patterns of concentrated face-centered cubic solid solutions have been widely attributed to chemical short-range order, although this connection has been recently questioned. This article explores the many non-ordering origins of commonly reported features using a combination of experimental electron microscopy and multislice diffraction simulations, which suggest that diffuse intensities largely represent thermal and static displacement scattering. A limited number of observations may reflect additional contributions from planar defects, surface terminations incommensurate with bulk periodicity, or weaker dynamical effectsComment: 8 pages, 3 figure

Extra electron reflections in concentrated alloys may originate from planar defects, not short-range order

In many concentrated alloys of current interest, the observation of diffuse superlattice intensities by transmission electron microscopy has been attributed to the presence of chemical short-range order. This interpretation is questioned on the basis of crystallographic considerations and theoretical predictions of ordering. The work of Xiao and Daykin [Ultramicroscopy 53 (1994)], which shows how planar defects can produce the exact set of observed peaks, is highlighted as an alternative explanation that would impact the conclusions of a number of recent studies.Comment: 5 pages, 3 figure

Alzheimer disease genetic risk factor APOE e4, and cognitive abilities in 111,739 UK Biobank participants

Background: the apolipoprotein (APOE) e4 locus is a genetic risk factor for dementia. Carriers of the e4 allele may be more vulnerable to conditions that are independent risk factors for cognitive decline, such as cardiometabolic diseases. Objective: we tested whether any association with APOE e4 status on cognitive ability was larger in older ages or in those with cardiometabolic diseases. Subjects: UK Biobank includes over 500,000 middle- and older aged adults who have undergone detailed medical and cognitive phenotypic assessment. Around 150,000 currently have genetic data. We examined 111,739 participants with complete genetic and cognitive data. Methods: baseline cognitive data relating to information processing speed, memory and reasoning were used. We tested for interactions with age and with the presence versus absence of type 2 diabetes (T2D), coronary artery disease (CAD) and hypertension. Results: in several instances, APOE e4 dosage interacted with older age and disease presence to affect cognitive scores. When adjusted for potentially confounding variables, there was no APOE e4 effect on the outcome variables. Conclusions: future research in large independent cohorts should continue to investigate this important question, which has potential implications for aetiology related to dementia and cognitive impairment

On the action potential as a propagating density pulse and the role of anesthetics

The Hodgkin-Huxley model of nerve pulse propagation relies on ion currents through specific resistors called ion channels. We discuss a number of classical thermodynamic findings on nerves that are not contained in this classical theory. Particularly striking is the finding of reversible heat changes, thickness and phase changes of the membrane during the action potential. Data on various nerves rather suggest that a reversible density pulse accompanies the action potential of nerves. Here, we attempted to explain these phenomena by propagating solitons that depend on the presence of cooperative phase transitions in the nerve membrane. These transitions are, however, strongly influenced by the presence of anesthetics. Therefore, the thermodynamic theory of nerve pulses suggests a explanation for the famous Meyer-Overton rule that states that the critical anesthetic dose is linearly related to the solubility of the drug in the membranes.Comment: 13 pages, 8 figure

The pain experiences of powered wheelchair users

Copyright © 2012 Informa UK, Ltd. This is the author's accepted manuscript. The final published article is available from the link below.Purpose: To explore the experience of pain and discomfort in users of electric-powered indoor/outdoor wheelchairs (EPIOCs) provided by a National Health Service. Methods: EPIOC users receiving their chair between February and November 2002 (N=74) were invited to participate in a telephone questionnaire/interview and 64 (aged 1081 years) agreed. Both specific and open-ended questions examined the presence of pain/discomfort, its severity, minimizing and aggravating factors, particularly in relation to the EPIOC and its use. Results: Most EPIOC users described experiences of pain with 17% reporting severe pain. Over half felt their pain was influenced by the wheelchair and few (25%) considered their chair eased their symptoms. The most common strategy for pain relief was taking medication. Other self-help strategies included changing position, exercise and complementary therapies. Respondents emphasized the provision of backrests, armrests, footrests and cushions which might alleviate or exacerbate pain, highlighting the importance of appropriate assessment for this high dependency group. Conclusions: Users related pain to their underlying medical condition, their wheelchair or a combination of the two. User feedback is essential to ensure that the EPIOC meets health needs with minimal pain. This becomes more important as the health condition of users changes over time

Hierarchy of modes in an interacting one-dimensional system.

Studying interacting fermions in one dimension at high energy, we find a hierarchy in the spectral weights of the excitations theoretically, and we observe evidence for second-level excitations experimentally. Diagonalizing a model of fermions (without spin), we show that levels of the hierarchy are separated by powers of R^{2}/L^{2}, where R is a length scale related to interactions and L is the system length. The first-level (strongest) excitations form a mode with parabolic dispersion, like that of a renormalized single particle. The second-level excitations produce a singular power-law line shape to the first-level mode and multiple power laws at the spectral edge. We measure momentum-resolved tunneling of electrons (fermions with spin) from or to a wire formed within a GaAs heterostructure, which shows parabolic dispersion of the first-level mode and well-resolved spin-charge separation at low energy with appreciable interaction strength. We find structure resembling the second-level excitations, which dies away quite rapidly at high momentum.We acknowledge financial support from the UK EPSRC through Grant No. EP/J01690X/1 and EP/J016888/1.This is the accepted manuscript. The final version is available at http://journals.aps.org/prl/abstract/10.1103/PhysRevLett.114.196401

Measuring axial length of the eye from magnetic resonance brain imaging

BACKGROUND: Metrics derived from the human eye are increasingly used as biomarkers and endpoints in studies of cardiovascular, cerebrovascular and neurological disease. In this context, it is important to account for potential confounding that can arise from differences in ocular dimensions between individuals, for example, differences in globe size. METHODS: We measured axial length, a geometric parameter describing eye size from T(2)-weighted brain MRI scans using three different image analysis software packages (Mango, ITK and Carestream) and compared results to biometry measurements from a specialized ophthalmic instrument (IOLMaster 500) as the reference standard. RESULTS: Ninety-three healthy research participants of mean age 51.0 ± SD 5.4 years were analyzed. The level of agreement between the MRI-derived measurements and the reference standard was described by mean differences as follows, Mango − 0.8 mm; ITK − 0.5 mm; and Carestream − 0.1 mm (upper/lower 95% limits of agreement across the three tools ranged from 0.9 mm to − 2.6 mm). Inter-rater reproducibility was between − 0.03 mm and 0.45 mm (ICC 0.65 to 0.93). Intra-rater repeatability was between 0.0 mm and − 0.2 mm (ICC 0.90 to 0.95). CONCLUSIONS: We demonstrate that axial measurements of the eye derived from brain MRI are within 3.5% of the reference standard globe length of 24.1 mm. However, the limits of agreement could be considered clinically significant. Axial length of the eye obtained from MRI is not a replacement for the precision of biometry, but in the absence of biometry it could provide sufficient accuracy to act as a proxy. We recommend measuring eye axial length from MRI in studies that do not have biometry but use retinal imaging to study neurodegenerative changes so as to control for differing eye size across individuals. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1186/s12886-022-02289-y