AmadeusGPT: a natural language interface for interactive animal behavioral analysis

The process of quantifying and analyzing animal behavior involves translating the naturally occurring descriptive language of their actions into machine-readable code. Yet, codifying behavior analysis is often challenging without deep understanding of animal behavior and technical machine learning knowledge. To limit this gap, we introduce AmadeusGPT: a natural language interface that turns natural language descriptions of behaviors into machine-executable code. Large-language models (LLMs) such as GPT3.5 and GPT4 allow for interactive language-based queries that are potentially well suited for making interactive behavior analysis. However, the comprehension capability of these LLMs is limited by the context window size, which prevents it from remembering distant conversations. To overcome the context window limitation, we implement a novel dual-memory mechanism to allow communication between short-term and long-term memory using symbols as context pointers for retrieval and saving. Concretely, users directly use language-based definitions of behavior and our augmented GPT develops code based on the core AmadeusGPT API, which contains machine learning, computer vision, spatio-temporal reasoning, and visualization modules. Users then can interactively refine results, and seamlessly add new behavioral modules as needed. We benchmark AmadeusGPT and show we can produce state-of-the-art performance on the MABE 2022 behavior challenge tasks. Note, an end-user would not need to write any code to achieve this. Thus, collectively AmadeusGPT presents a novel way to merge deep biological knowledge, large-language models, and core computer vision modules into a more naturally intelligent system. Code and demos can be found at: https://github.com/AdaptiveMotorControlLab/AmadeusGPT.Comment: demo available https://github.com/AdaptiveMotorControlLab/AmadeusGP

Object Recognition Using Convolutional Neural Networks

This chapter intends to present the main techniques for detecting objects within images. In recent years there have been remarkable advances in areas such as machine learning and pattern recognition, both using convolutional neural networks (CNNs). It is mainly due to the increased parallel processing power provided by graphics processing units (GPUs). In this chapter, the reader will understand the details of the state-of-the-art algorithms for object detection in images, namely, faster region convolutional neural network (Faster RCNN), you only look once (YOLO), and single shot multibox detector (SSD). We will present the advantages and disadvantages of each technique from a series of comparative tests. For this, we will use metrics such as accuracy, training difficulty, and characteristics to implement the algorithms. In this chapter, we intend to contribute to a better understanding of the state of the art in machine learning and convolutional networks for solving problems involving computational vision and object detection

Etude expérimentale des dynamiques temporelles du comportement normal et pathologique chez le rat et la souris

155 p.Modern neuroscience highlights the need for designing sophisticated behavioral readout of internal cognitive states. From a thorough analysis of classical behavioral test, my results supports the hypothesis that sensory ypersensitivity might be the cause of other behavioural deficits, and confirm the potassium channel BKCa as a potentially relevant molecular target for the development of drug medication against Fragile X Syndrome/Autism Spectrum Disorders. I have also used an innovative device, based on pressure sensors that can non-invasively detect the slightest animal movement with unprecedented sensitivity and time resolution, during spontaneous behaviour. Analysing this signal with sophisticated computational tools, I could demonstrate the outstanding potential of this methodology for behavioural phenotyping in general, and more specifically for the investigation of pain, fear or locomotion in normal mice and models of neurodevelopmental and neurodegenerative disorders

Trajectory Data Mining in Mouse Models of Stroke

Contains fulltext : 273912.pdf (Publisher’s version ) (Open Access)Radboud University, 04 oktober 2022Promotor : Kiliaan, A.J. Co-promotor : Wiesmann, M.167 p

Description and validation of the LocoWhisk system: Quantifying rodent exploratory, sensory and motor behaviours

Previous studies have demonstrated that analysing whisker movements and locomotion allows us to quantify the behavioural consequences of sensory, motor and cognitive deficits in rodents. Independent whisker and feet trackers exist but there is no fully-automated, open-source software and hardware solution, that measures both whisker movements and gait. New method We present the LocoWhisk arena and new accompanying software (ARTv2) that allows the automatic detection and measurement of both whisker and gait information from high-speed video footage. Results We demonstrate the new whisker and foot detector algorithms on high-speed video footage of freely moving small mammals, and show that whisker movement and gait measurements collected in the LocoWhisk arena are similar to previously reported values in the literature. Comparison with existing method(s) We demonstrate that the whisker and foot detector algorithms, are comparable in accuracy, and in some cases significantly better, than readily available software and manual trackers. Conclusion The LocoWhisk system enables the collection of quantitative data from whisker movements and locomotion in freely behaving rodents. The software automatically records both whisker and gait information and provides added statistical tools to analyse the data. We hope the LocoWhisk system and software will serve as a solid foundation from which to support future research in whisker and gait analysis

Real-time extensive livestock monitoring using lpwan smart wearable and infrastructure

Extensive unsupervised livestock farming is a habitual technique in many places around the globe. Animal release can be done for months, in large areas and with different species packing and behaving very differently. Nevertheless, the farmer’s needs are similar: where livestock is (and where has been) and how healthy they are. The geographical areas involved usually have difficult access with harsh orography and lack of communications infrastructure. This paper presents the design of a solution for extensive livestock monitoring in these areas. Our proposal is based in a wearable equipped with inertial sensors, global positioning system and wireless communications; and a Low-Power Wide Area Network infrastructure that can run with and without internet connection. Using adaptive analysis and data compression, we provide real-time monitoring and logging of cattle’s position and activities. Hardware and firmware design achieve very low energy consumption allowing months of battery life. We have thoroughly tested the devices in different laboratory setups and evaluated the system performance in real scenarios in the mountains and in the forest

Computational and Imaging Methods for Studying Neuronal Populations during Behavior

One of the central questions in neuroscience is how the nervous system generates and regulates behavior. To understand the neural code for any behavior, an ideal experiment would entail (i) quantitatively defining that behavior, (ii) recording neuronal activity in relevant brain regions to identify the underlying neuronal circuits and eventually (iii) manipulating them to test their function. Novel methods in neuroscience have greatly advanced our abilities to conduct such experiments but are still insufficient. In this thesis, I developed methods for these three goals. In Chapter 2, I describe an automatic behavior identification and classification method for the cnidarian Hydra vulgaris using machine learning. In Chapter 3, I describe a fast volumetric two-photon microscope with dual-color laser excitation that can image in 3D the activity of populations of neurons from visual cortex of awake mice. In Chapter 4, I present a machine learning method that identifies cortical ensembles and pattern completion neurons in mouse visual cortex, using two-photon calcium imaging data. These methods advance current technologies, providing opportunities for new discoveries

Functional trajectories during innate spinal cord repair

Adult zebrafish are capable of anatomical and functional recovery following severe spinal cord injury. Axon growth, glial bridging and adult neurogenesis are hallmarks of cellular regeneration during spinal cord repair. However, the correlation between these cellular regenerative processes and functional recovery remains to be elucidated. Whereas the majority of established functional regeneration metrics measure swim capacity, we hypothesize that gait quality is more directly related to neurological health. Here, we performed a longitudinal swim tracking study for 60 individual zebrafish spanning 8 weeks of spinal cord regeneration. Multiple swim parameters as well as axonal and glial bridging were integrated. We established rostral compensation as a new gait quality metric that highly correlates with functional recovery. Tensor component analysis of longitudinal data supports a correspondence between functional recovery trajectories and neurological outcomes. Moreover, our studies predicted and validated that a subset of functional regeneration parameters measured 1 to 2 weeks post-injury is sufficient to predict the regenerative outcomes of individual animals at 8 weeks post-injury. Our findings established new functional regeneration parameters and generated a comprehensive correlative database between various functional and cellular regeneration outputs