Hyperparameter Optimization and Boosting for Classifying Facial Expressions: How good can a “Null” Model be?

One of the goals of the ICML workshop on representation and learning is to establish benchmark scores for a new data set of labeled facial expressions. This paper presents the performance of a "Null" model consisting of convolutions with random weights, PCA, pooling, normalization, and a linear readout. Our approach focused on hyperparameter optimization rather than novel model components. On the Facial Expression Recognition Challenge held by the Kaggle website, our hyperparameter optimization approach achieved a score of 60% accuracy on the test data. This paper also introduces a new ensemble construction variant that combines hyperparameter optimization with the construction of ensembles. This algorithm constructed an ensemble of four models that scored 65.5% accuracy. These scores rank 12th and 5th respectively among the 56 challenge participants. It is worth noting that our approach was developed prior to the release of the data set, and applied without modification; our strong competition performance suggests that the TPE hyperparameter optimization algorithm and domain expertise encoded in our Null model can generalize to new image classification data sets.Engineering and Applied Science

Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures

Many computer vision algorithms depend on configuration settings that are typically hand-tuned in the course of evaluating the algorithm for a particular data set. While such parameter tuning is often presented as being incidental to the algorithm, correctly setting these parameter choices is frequently critical to realizing a method’s full potential. Compounding matters, these parameters often must be re-tuned when the algorithm is applied to a new problem domain, and the tuning process itself often depends on personal experience and intuition in ways that are hard to quantify or describe. Since the performance of a given technique depends on both the fundamental quality of the algorithm and the details of its tuning, it is sometimes difficult to know whether a given technique is genuinely better, or simply better tuned. In this work, we propose a meta-modeling approach to support automated hyperparameter optimization, with the goal of providing practical tools that replace hand-tuning with a reproducible and unbiased optimization process. Our approach is to expose the underlying expression graph of how a performance metric (e.g. classification accuracy on validation examples) is computed from hyperparameters that govern not only how individual processing steps are applied, but even which processing steps are included. A hyperparameter optimization algorithm transforms this graph into a program for optimizing that performance metric. Our approach yields state of the art results on three disparate computer vision problems: a face-matching verification task (LFW), a face identification task (PubFig83) and an object recognition task (CIFAR-10), using a single broad class of feed-forward vision architectures.Engineering and Applied Science

Reverse engineering object recognition

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Brain and Cognitive Sciences, 2007.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Page 95 blank.Includes bibliographical references (p. 83-94).Any given object in the world can cast an effectively infinite number of different images onto the retina, depending on its position relative to the viewer, the configuration of light sources, and the presence of other objects in the visual field. In spite of this, primates can robustly recognize a multitude of objects in a fraction of a second, with no apparent effort. The computational mechanisms underlying these amazing abilities are poorly understood. This thesis presents a collection of work from human psychophysics, monkey electrophysiology, and computational modelling in an effort to reverse-engineer the key computational components that enable this amazing ability in the primate visual system.by David Daniel Cox.Ph.D

Perceptual Annotation: Measuring Human Vision to Improve Computer Vision

For many problems in computer vision, human learners are considerably better than machines. Humans possess highly accurate internal recognition and learning mechanisms that are not yet understood, and they frequently have access to more extensive training data through a lifetime of unbiased experience with the visual world. We propose to use visual psychophysics to directly leverage the abilities of human subjects to build better machine learning systems. First, we use an advanced online psychometric testing platform to make new kinds of annotation data available for learning. Second, we develop a technique for harnessing these new kinds of information – “perceptual annotations” – for support vector machines. A key intuition for this approach is that while it may remain infeasible to dramatically increase the amount of data and high-quality labels available for the training of a given system, measuring the exemplar-by-exemplar difficulty and pattern of errors of human annotators can provide important information for regularizing the solution of the system at hand. A case study for the problem face detection demonstrates that this approach yields state-ofthe- art results on the challenging FDDB data set.Engineering and Applied SciencesMolecular and Cellular Biolog

Analysis of Control Strategies for Aircraft Flight Upset Recovery

This paper proposes a framework for studying the ability of a control strategy, consisting of a control law and a command law, to recover an aircraft from ight conditions that may extend beyond the normal ight envelope. This study was carried out (i) by evaluating time responses of particular ight upsets, (ii) by evaluating local stability over an equilibrium manifold that included stall, and (iii) by bounding the set in the state space from where the vehicle can be safely own to wings-level ight. These states comprise what will be called the safely recoverable ight envelope (SRFE), which is a set containing the aircraft states from where a control strategy can safely stabilize the aircraft. By safe recovery it is implied that the tran- sient response stays between prescribed limits before converging to a steady horizontal ight. The calculation of the SRFE bounds yields the worst-case initial state corresponding to each control strategy. This information is used to compare alternative recovery strategies, determine their strengths and limitations, and identify the most e ective strategy. In regard to the control law, the authors developed feedback feedforward laws based on the gain scheduling of multivariable controllers. In regard to the command law, which is the mechanism governing the exogenous signals driving the feed- forward component of the controller, we developed laws with a feedback structure that combines local stability and transient response considera- tions. The upset recovery of the Generic Transport Model, a sub-scale twin-engine jet vehicle developed by NASA Langley Research Center, is used as a case study

Cryo-Electron Tomography Elucidates the Molecular Architecture of Treponema pallidum, the Syphilis Spirochete

Cryo-electron tomography (CET) was used to examine the native cellular organization of Treponema pallidum, the syphilis spirochete. T. pallidum cells appeared to form flat waves, did not contain an outer coat and, except for bulges over the basal bodies and widening in the vicinity of flagellar filaments, displayed a uniform periplasmic space. Although the outer membrane (OM) generally was smooth in contour, OM extrusions and blebs frequently were observed, highlighting the structure’s fluidity and lack of attachment to underlying periplasmic constituents. Cytoplasmic filaments converged from their attachment points opposite the basal bodies to form arrays that ran roughly parallel to the flagellar filaments along the inner surface of the cytoplasmic membrane (CM). Motile treponemes stably attached to rabbit epithelial cells predominantly via their tips. CET revealed that T. pallidum cell ends have a complex morphology and assume at least four distinct morphotypes. Images of dividing treponemes and organisms shedding cell envelope-derived blebs provided evidence for the spirochete’s complex membrane biology. In the regions without flagellar filaments, peptidoglycan (PG) was visualized as a thin layer that divided the periplasmic space into zones of higher and lower electron densities adjacent to the CM and OM, respectively. Flagellar filaments were observed overlying the PG layer, while image modeling placed the PG-basal body contact site in the vicinity of the stator–P-collar junction. Bioinformatics and homology modeling indicated that the MotB proteins of T. pallidum, Treponema denticola, and Borrelia burgdorferi have membrane topologies and PG binding sites highly similar to those of their well-characterized Escherichia coli and Helicobacter pylori orthologs. Collectively, our results help to clarify fundamental differences in cell envelope ultrastructure between spirochetes and gram-negative bacteria. They also confirm that PG stabilizes the flagellar motor and enable us to propose that in most spirochetes motility results from rotation of the flagellar filaments against the PG

Simulations of Oligomeric Intermediates in Prion Diseases

We extend our previous stochastic cellular automata based model for areal aggregation of prion proteins on neuronal surfaces. The new anisotropic model allow us to simulate both strong beta-sheet and weaker attachment bonds between proteins. Constraining binding directions allows us to generate aggregate structures with the hexagonal lattice symmetry found in recently observed in vitro experiments. We argue that these constraints on rules may correspond to underlying steric constraints on the aggregation process. We find that monomer dominated growth of the areal aggregate is too slow to account for some observed doubling time-to-incubation time ratios inferred from data, and so consider aggregation dominated by relatively stable but non-infectious oligomeric intermediates. We compare a kinetic theory analysis of oligomeric aggregation to spatially explicit simulations of the process. We find that with suitable rules for misfolding of oligomers, possibly due to water exclusion by the surrounding aggregate, the resulting oligomeric aggregation model maps onto our previous monomer aggregation model. Therefore it can produce some of the same attractive features for the description of prion incubation time data. We propose experiments to test the oligomeric aggregation model.Comment: 8 pages, 10 figures For larger versions of several figures, see http://asaph.ucdavis.edu/~dmobley and click on the prion paper lin

Mimesis stories: composing new nature music for the shakuhachi

Nature is a widespread theme in much new music for the shakuhachi (Japanese bamboo flute). This article explores the significance of such music within the contemporary shakuhachi scene, as the instrument travels internationally and so becomes rooted in landscapes outside Japan, taking on the voices of new creatures and natural phenomena. The article tells the stories of five compositions and one arrangement by non-Japanese composers, first to credit composers’ varied and personal responses to this common concern and, second, to discern broad, culturally syncretic traditions of nature mimesis and other, more abstract, ideas about the naturalness of sounds and creative processes (which I call musical naturalism). Setting these personal stories and longer histories side by side reveals that composition creates composers (as much as the other way around). Thus it hints at much broader terrain: the refashioning of human nature at the confluence between cosmopolitan cultural circulations and contemporary encounters with the more-than-human world

Imaging Carbon Monoxide Emission in the Starburst Galaxy NGC 6000

We present measurements of carbon monoxide emission in the central region of the nearby starburst NGC 6000 taken with the Submillimeter Array. The J=2-1 transition of 12CO, 13CO, and C18O were imaged at a resolution of ~3''x2'' (450x300 pc). We accurately determine the dynamical center of NGC 6000 at R.A(J2000.0)=15h49m49.5s and dec(J2000.0)=-29d23'13'' which agrees with the peak of molecular emission position. The observed CO dynamics could be explained in the context of the presence of a bar potential affecting the molecular material, likely responsible for the strong nuclear concentration where more than 85% of the gas is located. We detect a kinematically detached component of dense molecular gas at relatively high velocity which might be fueling the star formation. A total nuclear dynamical mass of 7x10^9 Msun is derived and a total mass of gas of 4.6x10^8 Msun, yielding a Mgas/Mdyn~6%, similar to other previously studied barred galaxies with central starbursts. We determined the mass of molecular gas with the optically thin isotopologue C18O and we estimate a CO-to-H2 conversion factor X(CO)=0.4x10^20 cm-2/(K km s-1) in agreement with that determined in other starburst galaxies.Comment: 31 pages, 7 figures. Accepted for publication in Astronomical Journal