Accuracy vs. Energy: An Assessment of Bee Object Inference in Videos From On-Hive Video Loggers With YOLOv3, YOLOv4-Tiny, and YOLOv7-Tiny

A continuing trend in precision apiculture is to use computer vision methods to quantify characteristics of bee traffic in managed colonies at the hive\u27s entrance. Since traffic at the hive\u27s entrance is a contributing factor to the hive\u27s productivity and health, we assessed the potential of three open-source convolutional network models, YOLOv3, YOLOv4-tiny, and YOLOv7-tiny, to quantify omnidirectional traffic in videos from on-hive video loggers on regular, unmodified one- and two-super Langstroth hives and compared their accuracies, energy efficacies, and operational energy footprints. We trained and tested the models with a 70/30 split on a dataset of 23,173 flying bees manually labeled in 5819 images from 10 randomly selected videos and manually evaluated the trained models on 3600 images from 120 randomly selected videos from different apiaries, years, and queen races. We designed a new energy efficacy metric as a ratio of performance units per energy unit required to make a model operational in a continuous hive monitoring data pipeline. In terms of accuracy, YOLOv3 was first, YOLOv7-tiny—second, and YOLOv4-tiny—third. All models underestimated the true amount of traffic due to false negatives. YOLOv3 was the only model with no false positives, but had the lowest energy efficacy and highest operational energy footprint in a deployed hive monitoring data pipeline. YOLOv7-tiny had the highest energy efficacy and the lowest operational energy footprint in the same pipeline. Consequently, YOLOv7-tiny is a model worth considering for training on larger bee datasets if a primary objective is the discovery of non-invasive computer vision models of traffic quantification with higher energy efficacies and lower operational energy footprints

Ambient Electromagnetic Radiation as a Predictor of Honey Bee (\u3ci\u3eApis mellifera\u3c/i\u3e) Traffic in Linear and Non-Linear Regression: Numerical Stability, Physical Time and Energy Efficiency

Since bee traffic is a contributing factor to hive health and electromagnetic radiation has a growing presence in the urban milieu, we investigate ambient electromagnetic radiation as a predictor of bee traffic in the hive’s vicinity in an urban environment. To that end, we built two multi-sensor stations and deployed them for four and a half months at a private apiary in Logan, Utah, U.S.A. to record ambient weather and electromagnetic radiation. We placed two non-invasive video loggers on two hives at the apiary to extract omnidirectional bee motion counts from videos. The time-aligned datasets were used to evaluate 200 linear and 3,703,200 non-linear (random forest and support vector machine) regressors to predict bee motion counts from time, weather, and electromagnetic radiation. In all regressors, electromagnetic radiation was as good a predictor of traffic as weather. Both weather and electromagnetic radiation were better predictors than time. On the 13,412 time-aligned weather, electromagnetic radiation, and bee traffic records, random forest regressors had higher maximum R2 scores and resulted in more energy efficient parameterized grid searches. Both types of regressors were numerically stable

On Video Analysis of Omnidirectional Bee Traffic: Counting Bee Motions with Motion Detection and Image Classification

Omnidirectional bee traffic is the number of bees moving in arbitrary directions in close proximity to the landing pad of a given hive over a given period of time. Video bee traffic analysis has the potential to automate the assessment of omnidirectional bee traffic levels, which, in turn, may lead to a complete or partial automation of honeybee colony health assessment. In this investigation, we proposed, implemented, and partially evaluated a two-tier method for counting bee motions to estimate levels of omnidirectional bee traffic in bee traffic videos. Our method couples motion detection with image classification so that motion detection acts as a class-agnostic object location method that generates a set of regions with possible objects and each such region is classified by a class-specific classifier such as a convolutional neural network or a support vector machine or an ensemble of classifiers such as a random forest. The method has been, and is being iteratively field tested in BeePi monitors, multi-sensor electronic beehive monitoring systems, installed on live Langstroth beehives in real apiaries. Deployment of a BeePi monitor on top of a beehive does not require any structural modification of the beehive’s woodenware, and is not disruptive to natural beehive cycles. To ensure the replicability of the reported findings and to provide a performance benchmark for interested research communities and citizen scientists, we have made public our curated and labeled image datasets of 167,261 honeybee images and our omnidirectional bee traffic videos used in this investigation

On the Computability of Primitive Recursive Functions by Feedforward Artificial Neural Networks

We show that, for a primitive recursive function h(x, t), where x is a n-tuple of natural numbers and t is a natural number, there exists a feedforward artificial neural network (x, t), such that for any n-tuple of natural numbers z and a positive natural number m, the first m + 1 terms of the sequence {h(z, t)} are the same as the terms of the tuple ((z, 0), ... ,(z, m))

Application of Digital Particle Image Velocimetry to Insect Motion: Measurement of Incoming, Outgoing, and Lateral Honeybee Traffic

The well-being of a honeybee (Apis mellifera) colony depends on forager traffic. Consistent discrepancies in forager traffic indicate that the hive may not be healthy and require human intervention. Honeybee traffic in the vicinity of a hive can be divided into three types: incoming, outgoing, and lateral. These types constitute directional traffic, and are juxtaposed with omnidirectional traffic where bee motions are considered regardless of direction. Accurate measurement of directional honeybee traffic is fundamental to electronic beehive monitoring systems that continuously monitor honeybee colonies to detect deviations from the norm. An algorithm based on digital particle image velocimetry is proposed to measure directional traffic. The algorithm uses digital particle image velocimetry to compute motion vectors, analytically classifies them as incoming, outgoing, or lateral, and returns the classified vector counts as measurements of directional traffic levels. Dynamic time warping is used to compare the algorithm’s omnidirectional traffic curves to the curves produced by a previously proposed bee motion counting algorithm based on motion detection and deep learning and to the curves obtained from a human observer’s counts on four honeybee traffic videos (2976 video frames). The currently proposed algorithm not only approximates the human ground truth on par with the previously proposed algorithm in terms of omnidirectional bee motion counts but also provides estimates of directional bee traffic and does not require extensive training. An analysis of correlation vectors of consecutive image pairs with single bee motions indicates that correlation maps follow Gaussian distribution and the three-point Gaussian sub-pixel accuracy method appears feasible. Experimental evidence indicates it is reasonable to treat whole bees as tracers, because whole bee bodies and not parts thereof cause maximum motion. To ensure the replicability of the reported findings, these videos and frame-by-frame bee motion counts have been made public. The proposed algorithm is also used to investigate the incoming and outgoing traffic curves in a healthy hive on the same day and on different days on a dataset of 292 videos (216,956 video frames)

User Intent Communication in Robot-Assisted Shopping for the Blind

The research reported in this chapter describes our work on robot-assisted shopping for the blind. In our previous research, we developed RoboCart, a robotic shopping cart for the visually impaired (Gharpure, 2008; Kulyukin et al., 2008; Kulyukin et al., 2005). RoboCart's operation includes four steps: 1) the blind shopper (henceforth the shopper) selects

Natural language processing in the faq finder system: Results and prospects

Abstract This paper describes some recent results regarding the employment of natural language processing techniques in the FAQ FINDER system. FAQ FINDER is a natural language question-answering system that uses files of frequently-asked questions as its knowledge base. Unlike AI questionanswering systems that focus on the generation of new answers, FAQ FINDER retrieves existing ones found in frequently-asked question files. FAQ FINDER uses a combination of statistical and natural language techniques to match user questions against known question/answer pairs from FAQ files. We strove in our experiments to identify the contribution of these techniques to the overall success of the system. One unexpected result was that our parsing technique was not contributing as much to the system's performance as we expected. We discuss some of the reasons why this may have been the case, and describe further natural language processing research designed to address the system's current needs