Automated Beehive Surveillance Using Computer Vision

The number of honey bees entering and leaving a hive throughout the day is an important metric for beekeepers monitoring the health of their hives. Activity at the hive entrance depends on many factors such as weather, colony health, and presence of an adequate food supply. Some commercial systems that utilize infrared sensor gates at the hive’s entrance exist for accurately measuring honeybee traffic. In this thesis, a solution that is based on visual information obtained through videos taken in front of the hives is explored. This thesis provides details on the design and implementation of a beehive surveillance system, describing the algorithms and techniques used. The system operates in a realistic outdoor apiary environment, providing constant surveillance in real-time. Results obtained using different algorithms within the system are compared and methods for evaluating performance are discussed

Honeybee-based biohybrid system for landmine detection

This research was funded in part by NATO Science for Peace and Security (SPS) Programme, project number SPS 985355, “Biological Method (Bees) for Explosive Detection”.Legacy landmines in post-conflict areas are a non-discriminatory lethal hazard and can still be triggered decades after the conflict has ended. Efforts to detect these explosive devices are expensive, time-consuming, and dangerous to humans and animals involved. While methods such as metal detectors and sniffer dogs have successfully been used in humanitarian demining, more tools are required for both site surveying and accurate mine detection. Honeybees have emerged in recent years as efficient bioaccumulation and biomonitoring animals. The system reported here uses two complementary landmine detection methods: passive sampling and active search. Passive sampling aims to confirm the presence of explosive materials in a mine-suspected area by the analysis of explosive material brought back to the colony on honeybee bodies returning from foraging trips. Analysis is performed by light-emitting chemical sensors detecting explosives thermally desorbed from a preconcentrator strip. The active search is intended to be able to pinpoint the place where individual landmines are most likely to be present. Used together, both methods are anticipated to be useful in an end-to-end process for area surveying, suspected hazardous area reduction, and post-clearing internal and external quality control in humanitarian demining.Publisher PDFPeer reviewe

A Preliminary Study of Image Analysis for Parasite Detection on Honey Bees

International Conference Image Analysis and Recognition (ICIAR 2018, Póvoa de Varzim, Portugal

Accuracy vs. Energy: An Assessment of Bee Object Inference in Videos From On-Hive Video Loggers With YOLOv3, YOLOv4-Tiny, and YOLOv7-Tiny

A continuing trend in precision apiculture is to use computer vision methods to quantify characteristics of bee traffic in managed colonies at the hive\u27s entrance. Since traffic at the hive\u27s entrance is a contributing factor to the hive\u27s productivity and health, we assessed the potential of three open-source convolutional network models, YOLOv3, YOLOv4-tiny, and YOLOv7-tiny, to quantify omnidirectional traffic in videos from on-hive video loggers on regular, unmodified one- and two-super Langstroth hives and compared their accuracies, energy efficacies, and operational energy footprints. We trained and tested the models with a 70/30 split on a dataset of 23,173 flying bees manually labeled in 5819 images from 10 randomly selected videos and manually evaluated the trained models on 3600 images from 120 randomly selected videos from different apiaries, years, and queen races. We designed a new energy efficacy metric as a ratio of performance units per energy unit required to make a model operational in a continuous hive monitoring data pipeline. In terms of accuracy, YOLOv3 was first, YOLOv7-tiny—second, and YOLOv4-tiny—third. All models underestimated the true amount of traffic due to false negatives. YOLOv3 was the only model with no false positives, but had the lowest energy efficacy and highest operational energy footprint in a deployed hive monitoring data pipeline. YOLOv7-tiny had the highest energy efficacy and the lowest operational energy footprint in the same pipeline. Consequently, YOLOv7-tiny is a model worth considering for training on larger bee datasets if a primary objective is the discovery of non-invasive computer vision models of traffic quantification with higher energy efficacies and lower operational energy footprints

Automated Collection Of Honey Bee Hive Data Using The Raspberry Pi

In recent years beekeepers have faced significant losses to their populations of managed honey bees, a phenomenon known as Colony Collapse Disorder (CCD). Many researchers are studying CCD, attempting to determine its cause and how its effects can be mitigated. Some research efforts have focused on the analysis of bee hive audio and video recordings to better understand the behavior of bees and the health of the hive. To provide data for this research, it is important to have a means of capturing audio, video, and other sensor data, using a system that is reliable, inexpensive, and causes minimal disruption to the bees’ behavior. This thesis details the design and implementation of a data collection system, known as BeeMon, which is based around the Raspberry Pi. This system automatically captures sensor data and sends it to a remote server for analysis. With the ability to operate continuously in an outdoor apiary environment, it allows for constant, near real-time data collection. The results of several years of real world operation are discussed, as well as some research that has used the data collected

IndusBee 4.0 – Integrated Intelligent Sensory Systems for Advanced Bee Hive Instrumentation and Hive Keepers’ Assistance Systems

The importance of insects, and honey bees in particular, for our ecosystem is undisputed. Currently, environmental problems from pesticides to parasites endanger the well-being or even the existence of honey bee colonies and insects in general. This imposes an increasing load on skills and activities of hive keepers. Sensors, instrumentation, and machine learning offer solutions on the one hand to effectively instrument bee hives and on the other hand to provide efficient assistance systems for hive keepers. By advanced hive instrumentation and intelligent evaluation of the acquired information hives can be monitored more easily and with less intrusion. Like in other industrial disciplines, e.g., Industry 4.0, operation can move from scheduled to event driven activity. The development in Micro-Electrical-Mechanical- Systems and Internet-of-Things field in general allows to achieve affordable integrated monitoring solutions. However, not in all tasks a dedicated instrumentation of each hive is required, and mobile assistance systems and devices to be employed in a single instance for the whole apiary will complement the instrumentation activity and the overall approach of our IndusBee 4.0 research project. Examples of this category are, e.g., honey quality assessment tool as an extension of established hygrometers or a system for improved automation of the tedious and time consuming screening for the varroa infestation of hives. This paper provides a review of activities in the field and presents the current status of contributions to both lines of research in our IndusBee 4.0 research project. With regard to hive instrumentation, in addition to standard temperature, moisture, and weight monitoring, an approach of acoustical in-hive monitoring with automated decision making and notification implemented in-hive in a SmartComb has been pursued. Further, integrated gas sensors are currently added to the SmartComb to explore the in-hive detection of infestation and illness, e.g., (American) foulbrood. Visual flight hole inspection is successively explored by a separate system in or at the hive. With regard to hive keepers’ assistance systems, an approach automating the screening for the varroa infestation of hives was tackled first. Here, a cost-effective two step procedure, a first attention step for detecting candidate regions and a final classification step of these candidate regions, is applied. It is aspired to extend the approach to continuous in-hive varroa infestation monitoring. The integration of all information from hive instrumentation and assistance systems with data fusion and data analysis activities in apiary intelligence unit is aspired in the next step

Système complet d’acquisition vidéo, de suivi de trajectoires et de modélisation comportementale pour des environnements 3D naturellement encombrés : application à la surveillance apicole

This manuscript provides the basis for a complete chain of videosurveillence for naturally cluttered environments. In the latter, we identify and solve the wide spectrum of methodological and technological barriers inherent to : 1) the acquisition of video sequences in natural conditions, 2) the image processing problems, 3) the multi-target tracking ambiguities, 4) the discovery and the modeling of recurring behavioral patterns, and 5) the data fusion. The application context of our work is the monitoring of honeybees, and in particular the study of the trajectories bees in flight in front of their hive. In fact, this thesis is part a feasibility and prototyping study carried by the two interdisciplinary projects EPERAS and RISQAPI (projects undertaken in collaboration with INRA institute and the French National Museum of Natural History). It is for us, computer scientists, and for biologists who accompanied us, a completely new area of investigation for which the scientific knowledge, usually essential for such applications, are still in their infancy. Unlike existing approaches for monitoring insects, we propose to tackle the problem in the three-dimensional space through the use of a high frequency stereo camera. In this context, we detail our new target detection method which we called HIDS segmentation. Concerning the computation of trajectories, we explored several tracking approaches, relying on more or less a priori, which are able to deal with the extreme conditions of the application (e.g. many targets, small in size, following chaotic movements). Once the trajectories are collected, we organize them according to a given hierarchical data structure and apply a Bayesian nonparametric approach for discovering emergent behaviors within the colony of insects. The exploratory analysis of the trajectories generated by the crowded scene is performed following an unsupervised classification method simultaneously over different levels of semantic, and where the number of clusters for each level is not defined a priori, but rather estimated from the data only. This approach is has been validated thanks to a ground truth generated by a Multi-Agent System. Then we tested it in the context of real data.Ce manuscrit propose une approche méthodologique pour la constitution d’une chaîne complète de vidéosurveillance pour des environnements naturellement encombrés. Nous identifions et levons un certain nombre de verrous méthodologiques et technologiques inhérents : 1) à l’acquisition de séquences vidéo en milieu naturel, 2) au traitement d’images, 3) au suivi multi-cibles, 4) à la découverte et la modélisation de motifs comportementaux récurrents, et 5) à la fusion de données. Le contexte applicatif de nos travaux est la surveillance apicole, et en particulier, l’étude des trajectoires des abeilles en vol devant la ruche. De ce fait, cette thèse se présente également comme une étude de faisabilité et de prototypage dans le cadre des deux projets interdisciplinaires EPERAS et RISQAPI (projets menées en collaboration avec l’INRA Magneraud et le Muséum National d’Histoire Naturelle). Il s’agit pour nous informaticiens et pour les biologistes qui nous ont accompagnés, d’un domaine d’investigation totalement nouveau, pour lequel les connaissances métiers, généralement essentielles à ce genre d’applications, restent encore à définir. Contrairement aux approches existantes de suivi d’insectes, nous proposons de nous attaquer au problème dans l’espace à trois dimensions grâce à l’utilisation d’une caméra stéréovision haute fréquence. Dans ce contexte, nous détaillons notre nouvelle méthode de détection de cibles appelée segmentation HIDS. Concernant le calcul des trajectoires, nous explorons plusieurs approches de suivi de cibles, s’appuyant sur plus ou moins d’a priori, susceptibles de supporter les conditions extrêmes de l’application (e.g. cibles nombreuses, de petite taille, présentant un mouvement chaotique). Une fois les trajectoires collectées, nous les organisons selon une structure de données hiérarchique et mettons en œuvre une approche Bayésienne non-paramétrique pour la découverte de comportements émergents au sein de la colonie d’insectes. L’analyse exploratoire des trajectoires issues de la scène encombrée s’effectue par classification non supervisée, simultanément sur des niveaux sémantiques différents, et où le nombre de clusters pour chaque niveau n’est pas défini a priori mais est estimé à partir des données. Cette approche est dans un premier temps validée à l’aide d’une pseudo-vérité terrain générée par un Système Multi-Agents, puis dans un deuxième temps appliquée sur des données réelles

Impacts des polluants métalliques sur l'abeille : de la colonie au cerveau

Les abeilles sont des pollinisateurs essentiels. Une pléthore de facteurs de stress environnementaux, tels que les produits agrochimiques, a été identifiée comme contribuant à leur déclin mondial. En particulier, ces facteurs de stress altèrent les processus cognitifs impliqués dans les comportements fondamentaux. Jusqu'à présent, cependant, on ne sait pratiquement rien de l'impact de l'exposition à des métaux lourds, dont la toxicité est avérée chez de nombreux organismes. Pourtant, leurs émissions mondiales résultant des activités humaines ont élevé leurs concentrations bien au-dessus des niveaux naturels dans l'air, le sol, l'eau et la flore, exposant ainsi les abeilles à tous les stades de leur vie. Le but de ma thèse était d'examiner les effets de la pollution métallique sur l'abeille domestique en utilisant une approche multi-échelle, du cerveau à la colonie, en laboratoire et sur le terrain. J'ai d'abord observé que les abeilles exposées à une gamme de concentrations de trois métaux communs (arsenic, plomb et zinc) en laboratoire étaient incapables de percevoir et éviter des concentrations usuelles, néanmoins nocives, de ces métaux dans leur nourriture. J'ai ensuite exposé de façon chronique des colonies à des concentrations réalistes de plomb dans la nourriture et démontré que la consommation de ce métal altérait la cognition et le développement morphologique des abeilles. Comme les polluants métalliques se trouvent souvent dans des mélanges complexes dans l'environnement, j'ai exploré l'effet des cocktails de métaux, montrant que l'exposition au plomb, à l'arsenic ou au cuivre seul était suffisante pour ralentir l'apprentissage et perturber le rappel de la mémoire, et que les combinaisons de ces métaux induisaient des effets négatifs additifs sur ces deux processus cognitifs. J'ai finalement étudié l'impact de l'exposition naturelle aux polluants métalliques dans un environnement contaminé, en collectant des abeilles à proximité d'une ancienne mine d'or, et montré que les individus des populations les plus exposées aux métaux présentaient des capacités d'apprentissage et de mémoire plus faibles, et des altérations de leur développement conduisant à une réduction de la taille de leur cerveau. Une analyse plus systématique des abeilles non exposées a révélé une relation entre la taille de la tête, la morphométrie du cerveau et les performances d'apprentissage dans différentes tâches comportementales, suggérant que l'exposition aux polluants métalliques amplifie ces variations naturelles. Ainsi, mes résultats suggèrent que les abeilles domestiques sont incapables d'éviter l'exposition à des concentrations réalistes de métaux qui sont préjudiciables au développement et aux fonctions cognitives, et appellent à une révision des niveaux environnementaux considérés comme "sûrs". Ma thèse est la première analyse intégrée de l'impact de plusieurs polluants métalliques sur la cognition, la morphologie et l'organisation cérébrale chez l'abeille, et vise à encourager de nouvelles études sur la contribution de la pollution métallique dans le déclin signalé des abeilles, et plus généralement, des insectes.Honey bees are crucial pollinators. A plethora of environmental stressors, such as agrochemicals, have been identified as contributors to their global decline. Especially, these stressors impair cognitive processes involved in fundamental behaviours. So far however, virtually nothing is known about the impact of metal pollutants, despite their known toxicity to many organisms. Their worldwide emissions resulting from human activities have elevated their concentrations far above natural baselines in the air, soil, water and flora, exposing bees at all life stages. The aim of my thesis was to examine the effects of metallic pollution on honey bees using a multiscale approach, from brain to colonies, in laboratory and field conditions. I first observed that bees exposed to a range of concentrations of three common metals (arsenic, lead and zinc) in the laboratory were unable to perceive and avoid, low, yet harmful, field-realistic concentrations of those metals in their food. I then chronically exposed colonies to field-realistic concentrations of lead in food and demonstrated that consumption of this metal impaired bee cognition and morphological development, leading to smaller adult bees. As metal pollutants are often found in complex mixtures in the environment, I explored the effect of cocktails of metals, showing that exposure to lead, arsenic or copper alone was sufficient to slow down learning and disrupt memory retrieval, and that combinations of these metals induced additive negative effects on both cognitive processes. I finally investigated the impact of natural exposure to metal pollutants in a contaminated environment, by collecting bees in the vicinity of a former gold mine, and showed that individuals from populations most exposed to metals exhibited lower learning and memory abilities, and development impairments conducing to reduced brain size. A more systematic analysis of unexposed bees revealed a relationship between head size, brain morphometrics and learning performances in different behavioural tasks, suggesting that exposure to metal pollutants magnifies these natural variations. Hence, altogether, my results suggest that honey bees are unable to avoid exposure to field-realistic concentrations of metals that are detrimental to development and cognitive functions; and call for a revision of the environmental levels considered as 'safe'. My thesis is the first integrated analysis of the impact of several metal pollutants on bee cognition, morphology and brain structure, and should encourage further studies on the contribution of metal pollution in the reported decline of honey bees, and more generally, of insects

The ethology of honeybees (Apis mellifera) studied using accelerometer technology

While the significance of vibrational communication across insect taxa has been fairly well studied, the substrate-borne vibrations of honeybees remains largely unexplored. Within this thesis I have monitored honeybees with a new method, that of logging their short pulsed vibrations on the long term, and I have started the longstanding endeavour of underpinning the applications of it. The use of advanced spectral analysis and machine learning techniques as part of this new method has revealed exciting statistics that challenges previous expert’s interpretations. This work is comprised of three results chapters with the aim of determining (1) what can the in-situ monitoring of specific honeybee pulsed vibrations tell us about the status of a colony? (2) What long term statistics be can identified to help to disentangle the function of two specific pulses of vibrations? (3) How effective is honeycomb-embedded accelerometer technology at assessing the ethology of honeybee colonies? In the first results chapter, I explore the contributions of developing pupae and larvae to accelerometer datasets by monitoring brood frames isolated from the colony with embedded accelerometers. From this, I show that very little vibrational information is obtainable from capped brood using accelerometer. However, I am able to showcase the quantitation of specific vibrational waveforms that are indicative of brood emergence from the honeycomb. In the second results chapter, the automated detection of honeybee whooping signals was achieved with an 83% accuracy, revealing never-before-seen long-term statistics of vibration, once thought to be an inhibitory or food request signal. Statistics show that this pulse is very common, highly repeatable, occurs mainly at night with a distinct decrease towards midday, is correlated with the brood cycle, and can be elicited en masse as a startle response by bees following the gentle knocking of the hive. Through synchronisation of high-definition video and accelerometer data, the honeybee dorso ventral abdominal shaking (DVA) signal has been physically quantitated, for the first time, giving a one to-one association between behaviour and intra-comb vibrations. From this, a novel method for the continuous in-situ non-invasive automated detection was developed for a honeybee signal previously thought to have no vibratory component. I show that the signal is detected with high frequency and repeatability, occurring mostly at night with a minimum towards mid-afternoon; inverse to that of the signal's amplitude over an average day. An unprecedented increase in the cumulative amplitude of DVA signals occurs in the hours preceding and following a primary swarm. These statistics suggests that the DVA signal may have additional functions other than as a foraging activation signal, and that the amplitude of the signal might be indicative of the switching of its dual, and potentially multiple functions. This work has pioneered the use of accelerometer technology for the long-term monitoring of honeybee pulsed vibrations, making significant contributions to the emerging field of biotremology. The applications of this work, however, go far beyond the realms of the honeybee. The methods developed throughout this thesis could easily be adapted for the automated in-situ monitoring of biologically relevant vibroacoustics of multiple wild taxonomic groups, including wasps, termites, elephants and bats, as well as for replacing the need for visually compiled ethograms within lab-based manipulative experiments