Limbs detection and tracking of head-fixed mice for behavioral phenotyping using motion tubes and deep learning

The broad accessibility of affordable and reliable recording equipment and its relative ease of use has enabled neuroscientists to record large amounts of neurophysiological and behavioral data. Given that most of this raw data is unlabeled, great effort is required to adapt it for behavioral phenotyping or signal extraction, for behavioral and neurophysiological data, respectively. Traditional methods for labeling datasets rely on human annotators which is a resource and time intensive process, which often produce data that that is prone to reproducibility errors. Here, we propose a deep learning-based image segmentation framework to automatically extract and label limb movements from movies capturing frontal and lateral views of head-fixed mice. The method decomposes the image into elemental regions (superpixels) with similar appearance and concordant dynamics and stacks them following their partial temporal trajectory. These 3D descriptors (referred as motion cues) are used to train a deep convolutional neural network (CNN). We use the features extracted at the last fully connected layer of the network for training a Long Short Term Memory (LSTM) network that introduces spatio-temporal coherence to the limb segmentation. We tested the pipeline in two video acquisition settings. In the first, the camera is installed on the right side of the mouse (lateral setting). In the second, the camera is installed facing the mouse directly (frontal setting). We also investigated the effect of the noise present in the videos and the amount of training data needed, and we found that reducing the number of training samples does not result in a drop of more than 5% in detection accuracy even when as little as 10% of the available data is used for training

Système de suivi de mouvement

Le comportement des petits animaux est important pour les chercheurs scientifiques et précliniques; ils veulent connaître les effets des interventions sur leur vie naturelle. Pour les maladies humaines, les rongeurs sont utilisés comme modèles. L’étude du comportement des rongeurs permet d’identifier et de développer de nouveaux médicaments pour les troubles psychiatriques et neurologiques. La surveillance des animaux peut être traitée et un grand nombre de données traitées peuvent conduire à de meilleurs résultats de recherche dans un temps plus court. Ce mémoire présente le système de suivi du comportement des rongeurs basé sur des techniques de vision numérique. En vision numérique, la détection d’un sujet consiste à balayer et à rechercher un objet dans une image ou une vidéo (qui n’est qu’une séquence d’images), mais la localisation d’un objet dans des images successives d’une vidéo est appelée suivi. Pour trouver la position d’un sujet dans une image, nous avons utilisé la détection du sujet et le suivi, car le suivi peut aider lorsque la détection échoue et vice et versa. Avec cette approche, nous pouvons suivre et détecter tout type du sujet (souris, headstage, ou par exemple un ballon). Il n’y a pas de dépendance au type de caméra. Pour trouver un sujet dans une image, nous utilisons l’algorithme AdaBoost en ligne qui est un algorithme de suivi du sujet et l’algorithme de Canny qui est un algorithme de détection du sujet, puis nous vérifions les résultats. Si l’algorithme Adaboost en ligne n’a pas pu trouver le sujet, nous utilisons l’algorithme Canny pour le trouver. En comparant les résultats de notre approche avec les résultats des algorithmes AdaBoost en ligne et Canny séparément, nous avons constaté que notre approche permet de mieux trouver le sujet dans l’image que lorsque nous utilisons ces deux algorithmes séparément. Dans ce mémoire, nous décrirons les algorithmes de détection et de suivi du sujet.Small animal behavior is important for science and preclinical researchers; they want to know the effects of interventions in their natural life. For human diseases, rodents are used as models; studying rodent behavior is good for identifying and developing new drugs for psychiatric and neurological disorders. Animal monitoring can be processed and a large number of data can lead to better research result in a shorter time. This thesis introduces the rodents’ behavior tracking system based on computer vision techniques. In computer vision, object detection is scanning and searching for an object in an image or a video (which is just a sequence of images) but locating an object in successive frames of a video is called tracking. To find the position of an object in an image, we use object detection and object tracking together because tracking can help when detection fails and inversely. With this approach, we can track and detect any objects (mouse, headstage, or a ball). There is no dependency to the camera type. To find an object in an image we use the online AdaBoost algorithm, which is an object tracking algorithm and the Canny algorithm, which is an object detection algorithm together, then we check the results. If the online Adaboost algorithm could not find the object, we use the Canny algorithm to find the object. By comparing the results of our approach with the results of the online AdaBoost and Canny algorithms separately, we found that our approach can find the object in the image better than when we use these two algorithms separately. In this thesis, we will describe implemented object detection and tracking algorithms

Políticas de Copyright de Publicações Científicas em Repositórios Institucionais: O Caso do INESC TEC

A progressiva transformação das práticas científicas, impulsionada pelo desenvolvimento das novas Tecnologias de Informação e Comunicação (TIC), têm possibilitado aumentar o acesso à informação, caminhando gradualmente para uma abertura do ciclo de pesquisa. Isto permitirá resolver a longo prazo uma adversidade que se tem colocado aos investigadores, que passa pela existência de barreiras que limitam as condições de acesso, sejam estas geográficas ou financeiras. Apesar da produção científica ser dominada, maioritariamente, por grandes editoras comerciais, estando sujeita às regras por estas impostas, o Movimento do Acesso Aberto cuja primeira declaração pública, a Declaração de Budapeste (BOAI), é de 2002, vem propor alterações significativas que beneficiam os autores e os leitores. Este Movimento vem a ganhar importância em Portugal desde 2003, com a constituição do primeiro repositório institucional a nível nacional. Os repositórios institucionais surgiram como uma ferramenta de divulgação da produção científica de uma instituição, com o intuito de permitir abrir aos resultados da investigação, quer antes da publicação e do próprio processo de arbitragem (preprint), quer depois (postprint), e, consequentemente, aumentar a visibilidade do trabalho desenvolvido por um investigador e a respetiva instituição. O estudo apresentado, que passou por uma análise das políticas de copyright das publicações científicas mais relevantes do INESC TEC, permitiu não só perceber que as editoras adotam cada vez mais políticas que possibilitam o auto-arquivo das publicações em repositórios institucionais, como também que existe todo um trabalho de sensibilização a percorrer, não só para os investigadores, como para a instituição e toda a sociedade. A produção de um conjunto de recomendações, que passam pela implementação de uma política institucional que incentive o auto-arquivo das publicações desenvolvidas no âmbito institucional no repositório, serve como mote para uma maior valorização da produção científica do INESC TEC.The progressive transformation of scientific practices, driven by the development of new Information and Communication Technologies (ICT), which made it possible to increase access to information, gradually moving towards an opening of the research cycle. This opening makes it possible to resolve, in the long term, the adversity that has been placed on researchers, which involves the existence of barriers that limit access conditions, whether geographical or financial. Although large commercial publishers predominantly dominate scientific production and subject it to the rules imposed by them, the Open Access movement whose first public declaration, the Budapest Declaration (BOAI), was in 2002, proposes significant changes that benefit the authors and the readers. This Movement has gained importance in Portugal since 2003, with the constitution of the first institutional repository at the national level. Institutional repositories have emerged as a tool for disseminating the scientific production of an institution to open the results of the research, both before publication and the preprint process and postprint, increase the visibility of work done by an investigator and his or her institution. The present study, which underwent an analysis of the copyright policies of INESC TEC most relevant scientific publications, allowed not only to realize that publishers are increasingly adopting policies that make it possible to self-archive publications in institutional repositories, all the work of raising awareness, not only for researchers but also for the institution and the whole society. The production of a set of recommendations, which go through the implementation of an institutional policy that encourages the self-archiving of the publications developed in the institutional scope in the repository, serves as a motto for a greater appreciation of the scientific production of INESC TEC