Blockout: Dynamic Model Selection for Hierarchical Deep Networks

Most deep architectures for image classification--even those that are trained to classify a large number of diverse categories--learn shared image representations with a single model. Intuitively, however, categories that are more similar should share more information than those that are very different. While hierarchical deep networks address this problem by learning separate features for subsets of related categories, current implementations require simplified models using fixed architectures specified via heuristic clustering methods. Instead, we propose Blockout, a method for regularization and model selection that simultaneously learns both the model architecture and parameters. A generalization of Dropout, our approach gives a novel parametrization of hierarchical architectures that allows for structure learning via back-propagation. To demonstrate its utility, we evaluate Blockout on the CIFAR and ImageNet datasets, demonstrating improved classification accuracy, better regularization performance, faster training, and the clear emergence of hierarchical network structures

Glassiness and constrained dynamics of a short-range non-disordered spin model

We study the low temperature dynamics of a two dimensional short-range spin system with uniform ferromagnetic interactions, which displays glassiness at low temperatures despite the absence of disorder or frustration. The model has a dual description in terms of free defects subject to dynamical constraints, and is an explicit realization of the ``hierarchically constrained dynamics'' scenario for glassy systems. We give a number of exact results for the statics of the model, and study in detail the dynamical behaviour of one-time and two-time quantities. We also consider the role played by the configurational entropy, which can be computed exactly, in the relation between fluctuations and response.Comment: 10 pages, 9 figures; minor changes, references adde

Sub 20 nm Silicon Patterning and Metal Lift-Off Using Thermal Scanning Probe Lithography

The most direct definition of a patterning process' resolution is the smallest half-pitch feature it is capable of transferring onto the substrate. Here we demonstrate that thermal Scanning Probe Lithography (t-SPL) is capable of fabricating dense line patterns in silicon and metal lift-off features at sub 20 nm feature size. The dense silicon lines were written at a half pitch of 18.3 nm to a depth of 5 nm into a 9 nm polyphthalaldehyde thermal imaging layer by t-SPL. For processing we used a three-layer stack comprising an evaporated SiO2 hardmask which is just 2-3 nm thick. The hardmask is used to amplify the pattern into a 50 nm thick polymeric transfer layer. The transfer layer subsequently serves as an etch mask for transfer into silicon to a nominal depth of 60 nm. The line edge roughness (3 sigma) was evaluated to be less than 3 nm both in the transfer layer and in silicon. We also demonstrate that a similar three-layer stack can be used for metal lift-off of high resolution patterns. A device application is demonstrated by fabricating 50 nm half pitch dense nickel contacts to an InAs nanowire.Comment: 7 pages, 5 figures, to be published in JVST

Tweed in Martensites: A Potential New Spin Glass

We've been studying the ``tweed'' precursors above the martensitic transition in shape--memory alloys. These characteristic cross--hatched modulations occur for hundreds of degrees above the first--order shape--changing transition. Our two--dimensional model for this transition, in the limit of infinite elastic anisotropy, can be mapped onto a spin--glass Hamiltonian in a random field. We suggest that the tweed precursors are a direct analogy of the spin--glass phase. The tweed is intermediate between the high--temperature cubic phase and the low--temperature martensitic phase in the same way as the spin--glass phase can be intermediate between ferromagnet and antiferromagnet.Comment: 18 pages and four figures (included