Automating image analysis by annotating landmarks with deep neural networks

Image and video analysis is often a crucial step in the study of animal behavior and kinematics. Often these analyses require that the position of one or more animal landmarks are annotated (marked) in numerous images. The process of annotating landmarks can require a significant amount of time and tedious labor, which motivates the need for algorithms that can automatically annotate landmarks. In the community of scientists that use image and video analysis to study the 3D flight of animals, there has been a trend of developing more automated approaches for annotating landmarks, yet they fall short of being generally applicable. Inspired by the success of Deep Neural Networks (DNNs) on many problems in the field of computer vision, we investigate how suitable DNNs are for accurate and automatic annotation of landmarks in video datasets representative of those collected by scientists studying animals. Our work shows, through extensive experimentation on videos of hawkmoths, that DNNs are suitable for automatic and accurate landmark localization. In particular, we show that one of our proposed DNNs is more accurate than the current best algorithm for automatic localization of landmarks on hawkmoth videos. Moreover, we demonstrate how these annotations can be used to quantitatively analyze the 3D flight of a hawkmoth. To facilitate the use of DNNs by scientists from many different fields, we provide a self contained explanation of what DNNs are, how they work, and how to apply them to other datasets using the freely available library Caffe and supplemental code that we provide.https://arxiv.org/abs/1702.00583Published versio

Discovering useful parts for pose estimation in sparsely annotated datasets

Our work introduces a novel way to increase pose estimation accuracy by discovering parts from unannotated regions of training images. Discovered parts are used to generate more accurate appearance likelihoods for traditional part-based models like Pictorial Structures and its derivatives. Our experiments on images of a hawkmoth in flight show that our proposed approach significantly improves over existing work for this application, while also being more generally applicable. Our proposed approach localizes landmarks at least twice as accurately as a baseline based on a Mixture of Pictorial Structures (MPS) model. Our unique High-Resolution Moth Flight (HRMF) dataset is made publicly available with annotations.https://arxiv.org/abs/1605.00707Accepted manuscrip

Children’s episodic and generic reports of alleged abuse

With the present data, we explored the relations between the language of interviewer questions, children’s reports, and case and child characteristics in forensic interviews. Results clearly indicated that the type of questions posed by interviewers – either probing generic or episodic features of an event – was related to the specificity of information reported by children. Further, interviewers appeared to adjust their questioning strategies based on the frequency of the alleged abuse. Children alleging single instances of abuse were asked more episodic questions than those alleging multiple abuses. In contrast, children alleging multiple incidents of abuse were asked a greater proportion of generic questions. Given that investigators often seek forensically-relevant episodic information, it is recommended that training for investigators focus on recognition of prompt selection tendencies and developing strategies for posing non-suggestive, episodically focused questions

Direct lateral maneuvers in hawkmoths

ABSTRACTWe used videography to investigate direct lateral maneuvers, i.e. ‘sideslips’, of the hawkmoth Manduca sexta. M. sexta sideslip by rolling their entire body and wings to reorient their net force vector. During sideslip they increase net aerodynamic force by flapping with greater amplitude, (in both wing elevation and sweep), allowing them to continue to support body weight while rolled. To execute the roll maneuver we observed in sideslips, they use an asymmetric wing stroke; increasing the pitch of the roll-contralateral wing pair, while decreasing that of the roll-ipsilateral pair. They also increase the wing sweep amplitude of, and decrease the elevation amplitude of, the contralateral wing pair relative to the ipsilateral pair. The roll maneuver unfolds in a stairstep manner, with orientation changing more during downstroke than upstroke. This is due to smaller upstroke wing pitch angle asymmetries as well as increased upstroke flapping counter-torque from left-right differences in global reference frame wing velocity about the moth's roll axis. Rolls are also opposed by stabilizing aerodynamic moments from lateral motion, such that rightward roll velocity will be opposed by rightward motion. Computational modeling using blade-element approaches confirm the plausibility of a causal linkage between the previously mentioned wing kinematics and roll/sideslip. Model results also predict high degrees of axial and lateral damping. On the time scale of whole and half wing strokes, left-right wing pair asymmetries directly relate to the first, but not second, derivative of roll. Collectively, these results strongly support a roll-based sideslip with a high degree of roll damping in M. sexta.Summary: We show that hawkmoths fly sideways by rolling in the direction of movement, adding a left- or right-ward component to their net lift vector. The underlying roll maneuvers are produced from a suite of asymmetric wing kinematic changes and are heavily damped

Flight Mechanics and Control of Escape Manoeuvres in Hummingbirds. I. Flight Kinematics

Hummingbirds are nature’s masters of aerobatic manoeuvres. Previous research shows that hummingbirds and insects converged evolutionarily upon similar aerodynamic mechanisms and kinematics in hovering. Herein, we use three-dimensional kinematic data to begin to test for similar convergence of kinematics used for escape flight and to explore the effects of body size upon manoeuvring. We studied four hummingbird species in North America including two large species (magnificent hummingbird, Eugenes fulgens, 7.8 g, and blue-throated hummingbird, Lampornis clemenciae, 8.0 g) and two smaller species (broad-billed hummingbird, Cynanthus latirostris, 3.4 g, and black-chinned hummingbirds Archilochus alexandri, 3.1 g). Starting from a steady hover, hummingbirds consistently manoeuvred away from perceived threats using a drastic escape response that featured body pitch and roll rotations coupled with a large linear acceleration. Hummingbirds changed their flapping frequency and wing trajectory in all three degrees of freedom on a stroke-by-stroke basis, likely causing rapid and significant alteration of the magnitude and direction of aerodynamic forces. Thus it appears that the flight control of hummingbirds does not obey the ‘helicopter model’ that is valid for similar escape manoeuvres in fruit flies. Except for broad-billed hummingbirds, the hummingbirds had faster reaction times than those reported for visual feedback control in insects. The two larger hummingbird species performed pitch rotations and global-yaw turns with considerably larger magnitude than the smaller species, but roll rates and cumulative roll angles were similar among the four species

Flight Mechanics and Control of Escape Manoeuvres in Hummingbirds. II. Aerodynamic Force Production, Flight Control and Performance Limitations

The superior manoeuvrability of hummingbirds emerges from complex interactions of specialized neural and physiological processes with the unique flight dynamics of flapping wings. Escape manoeuvring is an ecologically relevant, natural behaviour of hummingbirds, from which we can gain understanding into the functional limits of vertebrate locomotor capacity. Here, we extend our kinematic analysis of escape manoeuvres from a companion paper to assess two potential limiting factors of the manoeuvring performance of hummingbirds: (1) muscle mechanical power output and (2) delays in the neural sensing and control system. We focused on the magnificent hummingbird (Eugenes fulgens, 7.8 g) and the black-chinned hummingbird (Archilochus alexandri, 3.1 g), which represent large and small species, respectively. We first estimated the aerodynamic forces, moments and the mechanical power of escape manoeuvres using measured wing kinematics. Comparing active-manoeuvring and passive-damping aerodynamic moments, we found that pitch dynamics were lightly damped and dominated by the effect of inertia, while roll dynamics were highly damped. To achieve observed closed-loop performance, pitch manoeuvres required faster sensorimotor transduction, as hummingbirds can only tolerate half the delay allowed in roll manoeuvres. Accordingly, our results suggested that pitch control may require a more sophisticated control strategy, such as those based on prediction. For the magnificent hummingbird, we estimated that escape manoeuvres required muscle mass-specific power 4.5 times that during hovering. Therefore, in addition to the limitation imposed by sensorimotor delays, muscle power could also limit the performance of escape manoeuvres

Polyaryl ethers and related polysiloxane copolymer molecular coatings preparation and radiation degrdation

Poly(arylene ether sulfones) comprise a class of materials known as engineering thermoplastics which have a variety of important applications. These polymers are tough, rigid materials with good mechanical properties over a wide temperature range, and they are processed by conventional methods into products typically having excellent hydrolytic, thermal, oxidative and dimensional stability. Wholly aromatic random copolymers of hydroquinone and biphenol with 4.4 prime dichlorodiphenyl sulfone were synthesized via mechanical nucleophilic displacement. Their structures were characterized and mechanical behavior studied. These tough, ductile copolymers show excellent radiation resistance to electron beam treatment and retain much of the mechanical properties up to at least 700 Mrads under argon