Uncovering the Structure of Animal Behavior via Deep Learning

Understanding how the brain generates behavior is a core goal of neuroscience. The need for tools to quantify naturalistic, freely-moving animal behavior has given rise to the nascent field of computational ethology. The work presented in this thesis describes the key contributions we have made to this field in the form of novel computational approaches for quantifying animal behavior. Natural behavior can be quantified at varying degrees of detail, ranging from coarse center-of-mass positional tracking to detailed motion capture of individual body parts. Until recently, it was not possible to perform full body motion capture in freely behaving animals without physical markers, specialized hardware or other experimental constraints. To address this, we developed LEAP (LEAP Estimates Animal Pose) a system for unconstrained markerless motion capture. Inspired by emerging deep learning-based approaches for human pose estimation, LEAP uses neural networks to predict body landmark locations from raw video frames. A core innovation in LEAP was to leverage lightweight neural networks to quickly specialize on new datasets with very little labeled examples which can be iteratively improved through human-in-the-loop training. LEAP was highly effective at tracking animals from flies and mice to fish and giraffes, but it was not designed for tracking multiple animals simultaneously. To address this, we developed its successor, SLEAP (Social LEAP) that explicitly models the problem of tracking multiple poses when animals are closely interacting. SLEAP was implemented from the ground up as a deep learning framework with infrastructure enabling custom network architectures, and multiple approaches to detection, grouping and tracking. We showed that SLEAP outperforms existing methods by 1-2 orders of magnitude in both accuracy and speed, enabling realtime multi-animal pose tracking which we demonstrate by implementing closed-loop optogenetic control of social behaviors. Finally, we apply these methods by developing a high-resolution behavioral monitoring setup to probe the structure of fly courtship behavior. We used SLEAP to track poses of freely interacting pairs of males and females while recording courtship song. Through experimental manipulations and computational modeling of the female response to male song, we found evidence for specific neural circuit mechanisms for multisensory integration across timescales

Fast animal pose estimation using deep neural networks

This dataset contains videos of freely moving fruit flies, as well as trained networks and body position estimates for all ~21 million frames. Download the README.txt file for a detailed description of this dataset's content. See the code repository (https://github.com/talmo/leap) for usage examples of these files.Recent work quantifying postural dynamics has attempted to define the repertoire of behaviors performed by an animal. However, a major drawback to these techniques has been their reliance on dimensionality reduction of images which destroys information about which parts of the body are used in each behavior. To address this issue, we introduce a deep learning-based method for pose estimation, LEAP (LEAP Estimates Animal Pose). LEAP automatically predicts the positions of animal body parts using a deep convolutional neural network with as little as 10 frames of labeled data for training. This framework consists of a graphical interface for interactive labeling of body parts and software for training the network and fast prediction on new data (1 hr to train, 185 Hz predictions). We validate LEAP using videos of freely behaving fruit flies (Drosophila melanogaster) and track 32 distinct points on the body to fully describe the pose of the head, body, wings, and legs with an error rate of <3% of the animal's body length. We recapitulate a number of reported findings on insect gait dynamics and show LEAP's applicability as the first step in unsupervised behavioral classification. Finally, we extend the method to more challenging imaging situations (pairs of flies moving on a mesh-like background) and movies from freely moving mice (Mus musculus) where we track the full conformation of the head, body, and limbs

Automated gesture tracking in head-fixed mice

The preparation consisting of a head-fixed mouse on a spherical or cylindrical treadmill offers unique advantages in a variety of experimental contexts. Head fixation provides the mechanical stability necessary for optical and electrophysiological recordings and stimulation. Additionally, it can be combined with virtual environments such as T-mazes, enabling these types of recording during diverse behaviors. New method, in this paper we present a low-cost, easy-to-build acquisition system, along with scalable computational methods to quantitatively measure behavior (locomotion and paws, whiskers, and tail motion patterns) in head-fixed mice locomoting on cylindrical or spherical treadmills. Existing methods, several custom supervised and unsupervised methods have been developed for measuring behavior in mice. However, to date there is no low-cost, turn-key, general-purpose, and scalable system for acquiring and quantifying behavior in mice. Results, we benchmark our algorithms against ground truth data generated either by manual labeling or by simpler methods of feature extraction. We demonstrate that our algorithms achieve good performance, both in supervised and unsupervised settings. Conclusions, we present a low-cost suite of tools for behavioral quantification, which serve as valuable complements to recording and stimulation technologies being developed for the head-fixed mouse preparation

Automated gesture tracking in head-fixed mice

The preparation consisting of a head-fixed mouse on a spherical or cylindrical treadmill offers unique advantages in a variety of experimental contexts. Head fixation provides the mechanical stability necessary for optical and electrophysiological recordings and stimulation. Additionally, it can be combined with virtual environments such as T-mazes, enabling these types of recording during diverse behaviors. New method, in this paper we present a low-cost, easy-to-build acquisition system, along with scalable computational methods to quantitatively measure behavior (locomotion and paws, whiskers, and tail motion patterns) in head-fixed mice locomoting on cylindrical or spherical treadmills. Existing methods, several custom supervised and unsupervised methods have been developed for measuring behavior in mice. However, to date there is no low-cost, turn-key, general-purpose, and scalable system for acquiring and quantifying behavior in mice. Results, we benchmark our algorithms against ground truth data generated either by manual labeling or by simpler methods of feature extraction. We demonstrate that our algorithms achieve good performance, both in supervised and unsupervised settings. Conclusions, we present a low-cost suite of tools for behavioral quantification, which serve as valuable complements to recording and stimulation technologies being developed for the head-fixed mouse preparation

O uso do aspirado de medula óssea de ilíaco em falhas ósseas de fêmures de camundongos: estudo experimental The use of inhaled bone marrow of ileum in bone failures of femurs of rats: experimental study

Os autores estudam a utilização de medula óssea em camundongos como estimulação da formação de calo ósseo. Foram utilizados dez camundongos adultos machos de linhagem isogênica gioto com peso de aproximadamente 250 gramas, e realizadas falhas ósseas na região distal do fêmur com alternância do lado direito e esquerdo, divididos em grupos A e B, sendo como controle camundongos com falha óssea isolado e com falhas ósseas com medula óssea colhida previamente de cada camundongo. Após análise qualitativa e quantitativa foi observado que o uso do aspirado de medula óssea não leva à estimulação da formação do calo ósseo e não há o aumento de processo inflamatório local.<br>The aim of this study is to analyze the bone marrow employment in rats to stimulate the bone callus formation. Ten adult rats were used, male, isogenic, gioto lineage, approximate weight of 250 grams. Bone failures were produced at femur distal portion, alternating the right and left sides, and they were divided in group A and B. The control was held in rats presenting an isolated bone failure or having their bone marrow previously collected After quantitative and qualitative analysis, it was observed that the bone marrow utilization does not lead to the bone callus formation and there isn't an increase in the local inflammation process

The neural basis for a persistent internal state in Drosophila females

Sustained changes in mood or action require persistent changes in neural activity, but it has been difficult to identify the neural circuit mechanisms that underlie persistent activity and contribute to long-lasting changes in behavior. Here, we show that a subset of Doublesex+ pC1 neurons in the Drosophila female brain, called pC1d/e, can drive minutes-long changes in female behavior in the presence of males. Using automated reconstruction of a volume electron microscopic (EM) image of the female brain, we map all inputs and outputs to both pC1d and pC1e. This reveals strong recurrent connectivity between, in particular, pC1d/e neurons and a specific subset of Fruitless+ neurons called aIPg. We additionally find that pC1d/e activation drives long-lasting persistent neural activity in brain areas and cells overlapping with the pC1d/e neural network, including both Doublesex+ and Fruitless+ neurons. Our work thus links minutes-long persistent changes in behavior with persistent neural activity and recurrent circuit architecture in the female brain