Neural Baby Talk

We introduce a novel framework for image captioning that can produce natural language explicitly grounded in entities that object detectors find in the image. Our approach reconciles classical slot filling approaches (that are generally better grounded in images) with modern neural captioning approaches (that are generally more natural sounding and accurate). Our approach first generates a sentence `template' with slot locations explicitly tied to specific image regions. These slots are then filled in by visual concepts identified in the regions by object detectors. The entire architecture (sentence template generation and slot filling with object detectors) is end-to-end differentiable. We verify the effectiveness of our proposed model on different image captioning tasks. On standard image captioning and novel object captioning, our model reaches state-of-the-art on both COCO and Flickr30k datasets. We also demonstrate that our model has unique advantages when the train and test distributions of scene compositions -- and hence language priors of associated captions -- are different. Code has been made available at: https://github.com/jiasenlu/NeuralBabyTalkComment: 12 pages, 7 figures, CVPR 201

Moments-based tight-binding calculations of local electronic structure in InAs/GaAs quantum dots for comparison to experimental measurements

Local electronic properties of InAs/GaAsInAs∕GaAs nanostructures are studied using a real-space moments method sp3d5s*sp3d5s* tight-binding approach. The order (N)(N) method is unique because it allows for accurate and highly resolved determination of local density of states that accounts for local strain, disorder, and defects, without diagonalization of the full tight-binding Hamiltonian. The effects of free surfaces and strain are first investigated by considering pure, cuboidal GaAs nanostructures. The quantum confinement in an embedded InAs quantum dot is then shown directly through the local densities of states projected on different atoms in the structure. The relationship between effective energy band gap and quantum dot size is mapped onto a simple equation. Finally, the real-space study is applied to quantum dot structures observed experimentally using scanning tunneling microscopy. Atomic positions are obtained from the images and used as input into the tight-binding calculations in order to study interfacial effects on the local electronic structure of real embedded quantum dots.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/87778/2/053109_1.pd

Insect visual sensitivity to long wavelengths enhances colour contrast of insects against vegetation

The sensitivity of animal photoreceptors to different wavelengths of light strongly influence the perceived visual contrast of objects in the environment. Outside of the human visual wavelength range, ultraviolet sensitivity in many species provides important and behaviourally relevant visual contrast between objects. However, at the opposite end of the spectrum, the potential advantage of red sensitivity remains unclear. We investigated the potential benefit of long wavelength sensitivity by modelling the visual contrast of a wide range of jewel beetle colours against flowers and leaves of their host plants to hypothetical insect visual systems. We find that the presence of a long wavelength sensitive photoreceptor increases estimated colour contrast, particularly of beetles against leaves. Moreover, under our model parameters, a trichromatic visual system with ultraviolet (λ(max) = 355 nm), short (λ(max) = 445 nm) and long (λ(max) = 600 nm) wavelength photoreceptors performed as well as a tetrachromatic visual system, which had an additional medium wavelength photoreceptor (λ(max) = 530 nm). When we varied λ(max) for the long wavelength sensitive receptor in a tetrachromatic system, contrast values between beetles, flowers and leaves were all enhanced with increasing λ(max) from 580 nm to at least 640 nm. These results suggest a potential advantage of red sensitivity in visual discrimination of insect colours against vegetation and highlight the potential adaptive value of long wavelength sensitivity in insects