Detecting wildlife in unmanned aerial systems imagery using convolutional neural networks trained with an automated feedback loop

Using automated processes to detect wildlife in uncontrolled outdoor imagery in the field of wildlife ecology is a challenging task. This is especially true in imagery provided by an Unmanned Aerial System (UAS), where the relative size of wildlife is small and visually similar to its background. This work presents an automated feedback loop which can be used to train convolutional neural networks with extremely unbalanced class sizes, which alleviates some of these challenges. This work utilizes UAS imagery collected by the Wildlife@Home project, which has employed citizen scientists and trained experts to go through collected UAS imagery and classify it. Classified data is used as inputs to convolutional neural networks (CNNs) which seek to automatically mark which areas of the imagery contain wildlife. The output of the CNN is then passed to a blob counter which returns a population estimate for the image. The feedback loop was developed to help train the CNNs to better differentiate between the wildlife and the visually similar background and deal with the disparate amount of wildlife training images versus background training images. Utilizing the feedback loop dramatically reduced population count error rates from previously published work, from +150% to −3.93% on citizen scientist data and +88% to +5.24% on expert data

Using Citizen Scientists To Inform Machine Learning Algorithms To Automate The Detection Of Species In Ecological Imagery

Modern data collection techniques used by ecologists has created a deluge of data that is becoming increasingly difficult to store, filter, and analyze in an efficient and timely manner. In just two summers, over 65,000 unmanned aerial system (UAS) images were collected, comprising the several terabytes (TB) of data that was reviewed by citizen scientists to generate inputs for machine learning algorithms. Uncontrolled conditions and the small size of target species relative to the background further increase the difficulty of manually cataloging the images. To assist with locating and identifying snow geese in the UAS images, a citizen science web portal was created as part of Wildlife@Home. It is demonstrated that aggregate citizen scientist observations are similar in quality to observations made by trained experts and can be used to train convolutional neural networks (CNN) to automate the detection of species in the imagery. Using a dataset comprising of the aggregate observations produces consistently better results than datasets consisting of observations from a single altitude, indicating that more numerous but slightly variable observations is preferable to more consistent but less numerous observations. The framework developed requires system administrators to manually run scripts to populate the database with new images; however, this framework can be extended to allow researchers to create their own projects, upload new images, and download data for CNN training

Training Convolutional Neural Networks Using An Automated Feedback Loop To Estimate The Population Of Avian Species

Using automated processes to detect wildlife in uncontrolled outdoor imagery in the field of wildlife ecology is challenging task. This is especially true in imagery provided by an Unmanned Aerial System (UAS), where the relative size of wildlife is small and visually similar to its background. In the UAS imagery collected by the Wildlife@Home project, the data is also extremely unbalanced, with less than 1% of area in the imagery being of wildlife. To tackle these challenges, the Wildlife@Home project has employed citizen scientists and trained experts to go through collected UAS imagery and classify it. Classified data are used as inputs to convolutional neural networks (CNNs) which seek to automatically mark which areas of the imagery contain wildlife. The output of the CNN is then passed to a blob counter which returns a population estimate for the image. A feedback loop was developed to help train the CNNs to better differentiate between the wildlife and the the visually similar background and deal with the disparate amount of wildlife training images versus background training images. When using the feedback loop and citizen scientist provided data, population estimates by the CNN and blob counter are within 3.93% of the manual count by the field biologists. When expert provided data is used the estimates are within 5.24%. This is improved from 150% and 88% error in previous work which did not employ a feedback loop for the citizen science and expert data, respectively. Citizen scientist data worked better than expert data in the current work potentially because a matching algorithm was used on the citizen scientist data but not the expert data