FreezeOut: Accelerate Training by Progressively Freezing Layers

The early layers of a deep neural net have the fewest parameters, but take up the most computation. In this extended abstract, we propose to only train the hidden layers for a set portion of the training run, freezing them out one-by-one and excluding them from the backward pass. Through experiments on CIFAR, we empirically demonstrate that FreezeOut yields savings of up to 20% wall-clock time during training with 3% loss in accuracy for DenseNets, a 20% speedup without loss of accuracy for ResNets, and no improvement for VGG networks. Our code is publicly available at https://github.com/ajbrock/FreezeOutComment: Extended Abstrac

More than symbioses : orchid ecology ; with examples from the Sydney Region

The Orchidaceae are one of the largest and most diverse families of flowering plants. Orchids grow as terrestrial, lithophytic, epiphytic or climbing herbs but most orchids native to the Sydney Region can be placed in one of two categories. The first consists of terrestrial, deciduous plants that live in fire-prone environments, die back seasonally to dormant underground root tubers, possess exclusively subterranean roots, which die off as the plants become dormant, and belong to the subfamily Orchidoideae. The second consists of epiphytic or lithophytic, evergreen plants that live in fire-free environments, either lack specialised storage structures or possess succulent stems or leaves that are unprotected from fire, possess aerial roots that grow over the surface of, or free of, the substrate, and which do not die off seasonally, and belong to the subfamily Epidendroideae. Orchid seeds are numerous and tiny, lacking cotyledons and endosperm and containing minimal nutrient reserves. Although the seeds of some species can commence germination on their own, all rely on infection by mycorrhizal fungi, which may be species-specific, to grow beyond the earliest stages of development. Many epidendroid orchids are viable from an early stage without their mycorrhizal fungi but most orchidoid orchids rely, at least to some extent, on their mycorrhizal fungi throughout their lives. Some are completely parasitic on their fungi and have lost the ability to photosynthesize. Some orchids parasitize highly pathogenic mycorrhizal fungi and are thus indirectly parasitic on other plants. Most orchids have specialised relationships with pollinating animals, with many species each pollinated by only one species of insect. Deceptive pollination systems, in which the plants provide no tangible reward to their pollinators, are common in the Orchidaceae. The most common form of deceit is food mimicry, while at least a few taxa mimic insect brood sites. At least six lineages of Australian orchids have independently evolved sexual deception. In this syndrome, a flower mimics the female of the pollinating insect species. Male insects are attracted to the flower and attempt to mate with it, and pollinate it in the process. Little is known of most aspects of the population ecology of orchids native to the Sydney Region, especially their responses to fire. Such knowledge would be very useful in informing decisions in wildlife management

Generative and Discriminative Voxel Modeling with Convolutional Neural Networks

When working with three-dimensional data, choice of representation is key. We explore voxel-based models, and present evidence for the viability of voxellated representations in applications including shape modeling and object classification. Our key contributions are methods for training voxel-based variational autoencoders, a user interface for exploring the latent space learned by the autoencoder, and a deep convolutional neural network architecture for object classification. We address challenges unique to voxel-based representations, and empirically evaluate our models on the ModelNet benchmark, where we demonstrate a 51.5% relative improvement in the state of the art for object classification.Comment: 9 pages, 5 figures, 2 table

SMASH: One-Shot Model Architecture Search through HyperNetworks

Designing architectures for deep neural networks requires expert knowledge and substantial computation time. We propose a technique to accelerate architecture selection by learning an auxiliary HyperNet that generates the weights of a main model conditioned on that model's architecture. By comparing the relative validation performance of networks with HyperNet-generated weights, we can effectively search over a wide range of architectures at the cost of a single training run. To facilitate this search, we develop a flexible mechanism based on memory read-writes that allows us to define a wide range of network connectivity patterns, with ResNet, DenseNet, and FractalNet blocks as special cases. We validate our method (SMASH) on CIFAR-10 and CIFAR-100, STL-10, ModelNet10, and Imagenet32x32, achieving competitive performance with similarly-sized hand-designed networks. Our code is available at https://github.com/ajbrock/SMAS

Tracking the effects of interactions on spinons in gapless Heisenberg chains

We consider the effects of interactions on spinon excitations in Heisenberg spin-1/2 chains. We compute the exact two-spinon part of the longitudinal structure factor of the infinite chain in zero field for all values of anisotropy in the gapless antiferromagnetic regime, via an exact algebraic approach. Our results allow us to quantitatively describe the behavior of these fundamental excitations throughout the observable continuum, for cases ranging from free to fully coupled chains, thereby explicitly mapping the effects of "turning on the interactions" in a strongly correlated system

Global Innovations in Measurement and Evaluation

We researched the latest developments in theory and practice in measurement and evaluation. And we found that new thinking, techniques, and technology are influencing and improving practice. This report highlights 8 developments that we think have the greatest potential to improve evaluation and programme design, and the careful collection and use of data. In it, we seek to inform and inspire—to celebrate what is possible, and encourage wider application of these ideas

Xp54 and related (DDX6-like) RNA helicases: roles in messenger RNP assembly, translation regulation and RNA degradation

The DEAD-box RNA helicase Xp54 is an integral component of the messenger ribonucleoprotein (mRNP) particles of Xenopus oocytes. In oocytes, several abundant proteins bind pre-mRNA transcripts to modulate nuclear export, RNA stability and translational fate. Of these, Xp54, the mRNA-masking protein FRGY2 and its activating protein kinase CK2α, bind to nascent transcripts on chromosome loops, whereas an Xp54-associated factor, RapA/B, binds to the mRNP complex in the cytoplasm. Over-expression, mutation and knockdown experiments indicate that Xp54 functions to change the conformation of mRNP complexes, displacing one subset of proteins to accommodate another. The sequence of Xp54 is highly conserved in a wide spectrum of organisms. Like Xp54, Drosophila Me31B and Caenorhabditis CGH-1 are required for proper meiotic development, apparently by regulating the translational activation of stored mRNPs and also for sorting certain mRNPs into germplasm-containing structures. Studies on yeast Dhh1 and mammalian rck/p54 have revealed a key role for these helicases in mRNA degradation and in earlier remodelling of mRNP for entry into translation, storage or decay pathways. The versatility of Xp54 and related helicases in modulating the metabolism of mRNAs at all stages of their lifetimes marks them out as key regulators of post-transcriptional gene expression

Laser microsculpting for the generation of robust diffractive security markings on the surface of metals

AbstractWe report the development of a laser-based process for the direct writing (‘microsculpting’) of unique security markings (reflective phase holograms) on the surface of metals. In contrast to the common approaches used for unique marking of the metal products and components, e.g., polymer holographic stickers which are attached to metals as an adhesive tape, our process enables the generation of the security markings directly onto the metal surface and thus overcomes the problems with tampering and biocompatibility which are typical drawbacks of holographic stickers. The process uses 35ns laser pulses of wavelength 355nm to generate optically-smooth deformations on the metal surface using a localised laser melting process. Security markings (holographic structures) on 304-grade stainless steel surface are fabricated, and their resulted optical performance is tested using a He–Ne laser beam of 632.8nm wavelength

Enhancement of reliability in condition monitoring techniques in wind turbines

The majority of electrical failures in wind turbines occur in the semiconductor components (IGBTs) of converters. To increase reliability and decrease the maintenance costs associated with this component, several health-monitoring methods have been proposed in the literature. Many laboratory-based tests have been conducted to detect the failure mechanisms of the IGBT in their early stages through monitoring the variations of thermo-sensitive electrical parameters. The methods are generally proposed and validated with a single-phase converter with an air-cored inductive or resistive load. However, limited work has been carried out considering limitations associated with measurement and processing of these parameters in a three-phase converter. Furthermore, looking at just variations of the module junction temperature will most likely lead to unreliable health monitoring as different failure mechanisms have their own individual effects on temperature variations of some, or all, of the electrical parameters. A reliable health monitoring system is necessary to determine whether the temperature variations are due to the presence of a premature failure or from normal converter operation. To address this issue, a temperature measurement approach should be independent from the failure mechanisms. In this paper, temperature is estimated by monitoring an electrical parameter particularly affected by different failure types. Early bond wire lift-off is detected by another electrical parameter that is sensitive to the progress of the failure. Considering two separate electrical parameters, one for estimation of temperature (switching off time) and another to detect the premature bond wire lift-off (collector emitter on-state voltage) enhance the reliability of an IGBT could increase the accuracy of the temperature estimation as well as premature failure detection