Embedded neural network for real-time animal behavior classification

Recent biological studies have focused on understanding animal interactions and welfare. To help biolo- gists to obtain animals’ behavior information, resources like wireless sensor networks are needed. More- over, large amounts of obtained data have to be processed off-line in order to classify different behaviors.There are recent research projects focused on designing monitoring systems capable of measuring someanimals’ parameters in order to recognize and monitor their gaits or behaviors. However, network unre- liability and high power consumption have limited their applicability.In this work, we present an animal behavior recognition, classification and monitoring system based ona wireless sensor network and a smart collar device, provided with inertial sensors and an embeddedmulti-layer perceptron-based feed-forward neural network, to classify the different gaits or behaviorsbased on the collected information. In similar works, classification mechanisms are implemented in aserver (or base station). The main novelty of this work is the full implementation of a reconfigurableneural network embedded into the animal’s collar, which allows a real-time behavior classification andenables its local storage in SD memory. Moreover, this approach reduces the amount of data transmittedto the base station (and its periodicity), achieving a significantly improving battery life. The system hasbeen simulated and tested in a real scenario for three different horse gaits, using different heuristics andsensors to improve the accuracy of behavior recognition, achieving a maximum of 81%.Junta de Andalucía P12-TIC-130

Feature Extraction and Random Forest to Identify Sheep Behavior from Accelerometer Data

Sensor technologies play an essential part in the agricultural community and many other scientific and commercial communities. Accelerometer signals and Machine Learning techniques can be used to identify and observe behaviours of animals without the need for an exhaustive human observation which is labour intensive and time consuming. This study employed random forest algorithm to identify grazing, walking, scratching, and inactivity (standing, resting) of 8 Hebridean ewes located in Cheshire, Shotwick in the UK. We gathered accelerometer data from a sensor device which was fitted on the collar of the animals. The selection of the algorithm was based on previous research by which random forest achieved the best results among other benchmark techniques. Therefore, in this study, more focus was given to feature engineering to improve prediction performance. Seventeen features from time and frequency domain were calculated from the accelerometer measurements and the magnitude of the acceleration. Feature elimination was utilised in which highly correlated ones were removed, and only nine out of seventeen features were selected. The algorithm achieved an overall accuracy of 99.43% and a kappa value of 98.66%. The accuracy for grazing, walking, scratching, and inactive was 99.08%, 99.13%, 99.90%, and 99.85%, respectively. The overall results showed that there is a significant improvement over previous methods and studies for all mutually exclusive behaviours. Those results are promising, and the technique could be further tested for future real-time activity recognition

An Embedded Deep Learning Computer Vision Method for Driver Distraction Detection

Driver distraction is a modern issue when operating automotive vehicles. It can lead to impaired driving and potential accidents. Detecting driver distraction most often relies on analyzing a photo or video of the driver being distracted. This involves complex deep learning models which often can only be ran on computers too powerful and expensive to implement into automobiles. This thesis presents a method of detecting driver distraction using computer vision methods within an embedded environment. By taking the deep learning architecture SqueezeNet, which is optimized for embedded deployment, and benchmarking it on a Jetson Nano embedded computer, this thesis demonstrates a viable method of detecting driver distraction in real time. The method shown here involves making slight modifications to SqueezeNet to be trained on the AUC Distracted Driver Dataset, yielding accuracies as high as 93% and speeds as high as 11 FPS when detecting distracted driving. This performance is similar, and/ or better when compared to larger, more complex deep learning models trained for similar driver distraction detection applications.Master of Science in EngineeringComputer Engineering, College of Engineering & Computer ScienceUniversity of Michigan-Dearbornhttp://deepblue.lib.umich.edu/bitstream/2027.42/170914/1/Benjamin Roytburd Final Thesis.pdfDescription of Benjamin Roytburd Final Thesis.pdf : Thesi

Tecnologias IoT para pastoreio e controlo de postura animal

The unwanted and adverse weeds that are constantly growing in vineyards, force wine producers to repeatedly remove them through the use of mechanical and chemical methods. These methods include machinery such as plows and brushcutters, and chemicals as herbicides to remove and prevent the growth of weeds both in the inter-row and under-vine areas. Nonetheless, such methods are considered very aggressive for vines, and, in the second case, harmful for the public health, since chemicals may remain in the environment and hence contaminate water lines. Moreover, such processes have to be repeated over the year, making it extremely expensive and toilsome. Using animals, usually ovines, is an ancient practice used around the world. Animals, grazing in vineyards, feed from the unwanted weeds and fertilize the soil, in an inexpensive, ecological and sustainable way. However, sheep may be dangerous to vines since they tend to feed on grapes and on the lower branches of the vines, which causes enormous production losses. To overcome that issue, sheep were traditionally used to weed vineyards only before the beginning of the growth cycle of grapevines, thus still requiring the use of mechanical and/or chemical methods during the remainder of the production cycle. To mitigate the problems above, a new technological solution was investigated under the scope of the SheepIT project and developed in the scope of this thesis. The system monitors sheep during grazing periods on vineyards and implements a posture control mechanism to instruct them to feed only from the undesired weeds. This mechanism is based on an IoT architecture, being designed to be compact and energy efficient, allowing it to be carried by sheep while attaining an autonomy of weeks. In this context, the thesis herein sustained states that it is possible to design an IoT-based system capable of monitoring and conditioning sheep’s posture, enabling a safe weeding process in vineyards. Moreover, we support such thesis in three main pillars that match the main contributions of this work and that are duly explored and validated, namely: the IoT architecture design and required communications, a posture control mechanism and the support for a low-cost and low-power localization mechanism. The system architecture is validated mainly in simulation context while the posture control mechanism is validated both in simulations and field experiments. Furthermore, we demonstrate the feasibility of the system and the contribution of this work towards the first commercial version of the system.O constante crescimento de ervas infestantes obriga os produtores a manter um processo contínuo de remoção das mesmas com recurso a mecanismos mecânicos e/ou químicos. Entre os mais populares, destacam-se o uso de arados e roçadores no primeiro grupo, e o uso de herbicidas no segundo grupo. No entanto, estes mecanismos são considerados agressivos para as videiras, assim como no segundo caso perigosos para a saúde pública, visto que os químicos podem permanecer no ambiente, contaminando frutos e linhas de água. Adicionalmente, estes processos são caros e exigem mão de obra que escasseia nos dias de hoje, agravado pela necessidade destes processos necessitarem de serem repetidos mais do que uma vez ao longo do ano. O uso de animais, particularmente ovelhas, para controlar o crescimento de infestantes é uma prática ancestral usada em todo o mundo. As ovelhas, enquanto pastam, controlam o crescimento das ervas infestantes, ao mesmo tempo que fertilizam o solo de forma gratuita, ecológica e sustentável. Não obstante, este método foi sendo abandonado visto que os animais também se alimentam da rama, rebentos e frutos da videira, provocando naturais estragos e prejuízos produtivos. Para mitigar este problema, uma nova solução baseada em tecnologias de Internet das Coisas é proposta no âmbito do projeto SheepIT, cuja espinha dorsal foi construída no âmbito desta tese. O sistema monitoriza as ovelhas enquanto estas pastoreiam nas vinhas, e implementam um mecanismo de controlo de postura que condiciona o seu comportamento de forma a que se alimentem apenas das ervas infestantes. O sistema foi incorporado numa infraestrutura de Internet das Coisas com comunicações sem fios de baixo consumo para recolha de dados e que permite semanas de autonomia, mantendo os dispositivos com um tamanho adequado aos animais. Neste contexto, a tese suportada neste trabalho defende que é possível projetar uma sistema baseado em tecnologias de Internet das Coisas, capaz de monitorizar e condicionar a postura de ovelhas, permitindo que estas pastem em vinhas sem comprometer as videiras e as uvas. A tese é suportada em três pilares fundamentais que se refletem nos principais contributos do trabalho, particularmente: a arquitetura do sistema e respetivo sistema de comunicações; o mecanismo de controlo de postura; e o suporte para implementação de um sistema de localização de baixo custo e baixo consumo energético. A arquitetura é validada em contexto de simulação, e o mecanismo de controlo de postura em contexto de simulação e de experiências em campo. É também demonstrado o funcionamento do sistema e o contributo deste trabalho para a conceção da primeira versão comercial do sistema.Programa Doutoral em Informátic

Evaluación y análisis de una aproximación a la fusión sensorial neuronal mediante el uso de sensores pulsantes de visión / audio y redes neuronales de convolución

En este trabajo se pretende avanzar en el conocimiento y posibles implementaciones hardware de los mecanismos de Deep Learning, así como el uso de la fusión sensorial de forma eficiente utilizando dichos mecanismos. Para empezar, se realiza un análisis y estudio de los lenguajes de programación paralela actuales, así como de los mecanismos de Deep Learning para la fusión sensorial de visión y audio utilizando sensores neuromórficos para el uso en plataformas de FPGA. A partir de estos estudios, se proponen en primer lugar soluciones implementadas en OpenCL así como en hardware dedicado, descrito en systemverilog, para la aceleración de algoritmos de Deep Learning comenzando con el uso de un sensor de visión como entrada. Se analizan los resultados y se realiza una comparativa entre ellos. A continuación se añade un sensor de audio y se proponen mecanismos estadísticos clásicos, que sin ofrecer capacidad de aprendizaje, permiten integrar la información de ambos sensores, analizando los resultados obtenidos junto con sus limitaciones. Como colofón de este trabajo, para dotar al sistema de la capacidad de aprendizaje, se utilizan mecanismos de Deep Learning, en particular las CNN1, para fusionar la información audiovisual y entrenar el modelo para desarrollar una tarea específica. Al final se evalúa el rendimiento y eficiencia de dichos mecanismos obteniendo conclusiones y unas proposiciones de mejora que se dejarán indicadas para ser implementadas como trabajos futuros.In this work it is intended to advance on the knowledge and possible hardware implementations of the Deep Learning mechanisms, as well as on the use of sensory fusión efficiently using such mechanisms. At the beginning, it is performed an analysis and study of the current parallel programing, furthermore of the Deep Learning mechanisms for audiovisual sensory fusion using neuromorphic sensor on FPGA platforms. Based on these studies, first of all it is proposed solution implemented on OpenCL as well as dedicated hardware, described on systemverilog, for the acceleration of Deep Learning algorithms, starting with the use of a vision sensor as input. The results are analysed and a comparison between them has been made. Next, an audio sensor is added and classic statistical mechanisms are proposed, which, without providing learning capacity, allow the integration of information from both sensors, analysing the results obtained along with their limitations. Finally, in order to provide the system with learning capacity, Deep Learning mechanisms, in particular CNN, are used to merge audiovisual information and train the model to develop a specific task. In the end, the performance and efficiency of these mechanisms have been evaluated, obtaining conclusions and proposing improvements that will be indicated to be implemented as future works

Intelligent strategies for sheep monitoring and management

With the growth in world population, there is an increasing demand for food resources and better land utilisation, e.g., domesticated animals and land management, which in turn brought about developments in intelligent farming. Modern farms rely upon intelligent sensors and advanced software solutions, to optimally manage pasture and support animal welfare. A very significant aspect in domesticated animal farms is monitoring and understanding of animal activity, which provides vital insight into animal well-being and the environment they live in. Moreover, “virtual” fencing systems provide an alternative to managing farmland by replacing traditional boundaries. This thesis proposes novel solutions to animal activity recognition based on accelerometer data using machine learning strategies, and supports the development of virtual fencing systems via animal behaviour management using audio stimuli. The first contribution of this work is four datasets comprising accelerometer gait signals. The first dataset consisted of accelerometer and gyroscope measurements, which were obtained using a Samsung smartphone on seven animals. Next, a dataset of accelerometer measurements was collected using the MetamotionR device on 8 Hebridean ewes. Finally, two datasets of nine Hebridean ewes were collected from two sensors (MetamotionR and Raspberry Pi) comprising of accelerometer signals describing active, inactive and grazing activity of the animal. These datasets will be made publicly available as there is limited availability of such datasets. In respect to activity recognition, a systematic study of the experimental setup, associated signal features and machine learning methods was performed. It was found that Random Forest using accelerometer measurements and a sample rate of 12.5Hz with a sliding window of 5 seconds provides an accuracy of above 96% when discriminating animal activity. The problem of sensor heterogeneity was addressed with transfer learning of Convolutional Neural Networks, which has been used for the first time in this problem, and resulted to an accuracy of 98.55%, and 96.59%, respectively, in the two experimental datasets. Next, the feasibility of using only audio stimuli in the context of a virtual fencing system was explored. Specifically, a systematic evaluation of the parameters of audio stimuli, e.g., frequency and duration, was performed on two sheep breeds, Hebridean and Greyface Dartmoor ewes, in the context of controlling animal position and keeping them away from a designated area. It worth noting that the use of sounds is different to existing approaches, which utilize electric shocks to train animals to adhere within the boundaries of a virtual fence. It was found that audio signals in the frequencies of 125Hz-440Hz, 10kHz-17kHz and white noise are able to control animal activity with accuracies of 89.88%, and 95.93%, for Hebridean and Greyface Dartmoor ewes, respectively. Last but not least, the thesis proposes a multifunctional system that identifies whether the animal is active or inactive, using transfer learning, and manipulates its position using the optimized sound settings achieving a classification accuracy of over 99.95%