Wildbook: Crowdsourcing, computer vision, and data science for conservation

Photographs, taken by field scientists, tourists, automated cameras, and incidental photographers, are the most abundant source of data on wildlife today. Wildbook is an autonomous computational system that starts from massive collections of images and, by detecting various species of animals and identifying individuals, combined with sophisticated data management, turns them into high resolution information database, enabling scientific inquiry, conservation, and citizen science. We have built Wildbooks for whales (flukebook.org), sharks (whaleshark.org), two species of zebras (Grevy's and plains), and several others. In January 2016, Wildbook enabled the first ever full species (the endangered Grevy's zebra) census using photographs taken by ordinary citizens in Kenya. The resulting numbers are now the official species census used by IUCN Red List: http://www.iucnredlist.org/details/7950/0. In 2016, Wildbook partnered up with WWF to build Wildbook for Sea Turtles, Internet of Turtles (IoT), as well as systems for seals and lynx. Most recently, we have demonstrated that we can now use publicly available social media images to count and track wild animals. In this paper we present and discuss both the impact and challenges that the use of crowdsourced images can have on wildlife conservation.Comment: Presented at the Data For Good Exchange 201

Automatic Individual Identification of Patterned Solitary Species Based on Unlabeled Video Data

The manual processing and analysis of videos from camera traps is time-consuming and includes several steps, ranging from the filtering of falsely triggered footage to identifying and re-identifying individuals. In this study, we developed a pipeline to automatically analyze videos from camera traps to identify individuals without requiring manual interaction. This pipeline applies to animal species with uniquely identifiable fur patterns and solitary behavior, such as leopards (Panthera pardus). We assumed that the same individual was seen throughout one triggered video sequence. With this assumption, multiple images could be assigned to an individual for the initial database filling without pre-labeling. The pipeline was based on well-established components from computer vision and deep learning, particularly convolutional neural networks (CNNs) and scale-invariant feature transform (SIFT) features. We augmented this basis by implementing additional components to substitute otherwise required human interactions. Based on the similarity between frames from the video material, clusters were formed that represented individuals bypassing the open set problem of the unknown total population. The pipeline was tested on a dataset of leopard videos collected by the Pan African Programme: The Cultured Chimpanzee (PanAf) and achieved a success rate of over 83% for correct matches between previously unknown individuals. The proposed pipeline can become a valuable tool for future conservation projects based on camera trap data, reducing the work of manual analysis for individual identification, when labeled data is unavailable

Perspectives in machine learning for wildlife conservation

Data acquisition in animal ecology is rapidly accelerating due to inexpensive and accessible sensors such as smartphones, drones, satellites, audio recorders and bio-logging devices. These new technologies and the data they generate hold great potential for large-scale environmental monitoring and understanding, but are limited by current data processing approaches which are inefficient in how they ingest, digest, and distill data into relevant information. We argue that machine learning, and especially deep learning approaches, can meet this analytic challenge to enhance our understanding, monitoring capacity, and conservation of wildlife species. Incorporating machine learning into ecological workflows could improve inputs for population and behavior models and eventually lead to integrated hybrid modeling tools, with ecological models acting as constraints for machine learning models and the latter providing data-supported insights. In essence, by combining new machine learning approaches with ecological domain knowledge, animal ecologists can capitalize on the abundance of data generated by modern sensor technologies in order to reliably estimate population abundances, study animal behavior and mitigate human/wildlife conflicts. To succeed, this approach will require close collaboration and cross-disciplinary education between the computer science and animal ecology communities in order to ensure the quality of machine learning approaches and train a new generation of data scientists in ecology and conservation

Advances in automated wildlife photo-identification

La foto-identificación de especies silvestres es un recurso base para la obtención de información necesaria para diversas tareas de investigaciones biológicas. Hoy en día el crowdsourcing y la ciencia ciudadana están comenzando a desempeñar un rol importante en la recopilación de datos científicos. Esta fuente de datos permite aumentar considerablemente el número de registros en la base de muestreo de diferentes proyectos científicos, especialmente los relacionados con los modelos de capturarecaptura fotográfica de vida silvestre. No obstante, mientras que se aumenta la cantidad de datos recopilados de fuentes no científicas, se presenta un nuevo desafío, el procesamiento masivo de manera ágil y eficiente, que permita limpiar y seleccionar los datos relevantes para las siguientes etapas. Este trabajo aborda la automatización de la primer etapa del proceso de foto-identificación de cetáceos, el cual se trata de la detección de la presencia o ausencia de la región de interés en la imagen (ROI). Para ello, se especializó una red neuronal convolucional de propósito general (Mask R-CNN) con imágenes de delfines de la especie Cephalorhynchus commersonii recolectadas en diferentes sitios de la costa patagónica durante un período de siete años.The wild species photo-identification is a basic resource for obtaining the necessary information for several biological research tasks. Today crowdsourcing and citizen science are beginning to play an important role in collecting scientific data. This data source makes it possible to considerably increase the number of records in the sampling database for different scientific projects, especially those related to photographic capture-recapture models of wildlife. However, while the amount of data collected from non-scientific sources increases, a new challenge is presented, mass processing in an agile and efficient way, which allows cleaning and selecting the relevant data for the next stages. This work addresses the automation of the first stage of the cetacean photo-identification process, which is the detection of the presence or absence of the region of interest in the image (ROI). For this aim, a general-purpose convolutional neural network (Mask R-CNN) was specialized with dolphins images of Cephalorhynchus commersonii specie, collected at different sites on the Patagonian coast over a period of seven years.Fil: Pollicelli, Maria Debora. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Centro Nacional Patagónico. Centro para el Estudio de Sistemas Marinos; ArgentinaFil: Delrieux, Claudio Augusto. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Bahía Blanca; ArgentinaFil: Coscarella, Mariano Alberto. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaV Jornadas de Intercambio y Difusión de los Resultados de Investigaciones de los Doctorandos en IngenieríaBuenos AiresArgentinaUniversidad Tecnológica Naciona

The Caltech Fish Counting Dataset: A Benchmark for Multiple-Object Tracking and Counting

We present the Caltech Fish Counting Dataset (CFC), a large-scale dataset for detecting, tracking, and counting fish in sonar videos. We identify sonar videos as a rich source of data for advancing low signal-to-noise computer vision applications and tackling domain generalization in multiple-object tracking (MOT) and counting. In comparison to existing MOT and counting datasets, which are largely restricted to videos of people and vehicles in cities, CFC is sourced from a natural-world domain where targets are not easily resolvable and appearance features cannot be easily leveraged for target re-identification. With over half a million annotations in over 1,500 videos sourced from seven different sonar cameras, CFC allows researchers to train MOT and counting algorithms and evaluate generalization performance at unseen test locations. We perform extensive baseline experiments and identify key challenges and opportunities for advancing the state of the art in generalization in MOT and counting.Comment: ECCV 2022. 33 pages, 12 figure

A narrative review on the use of camera traps and machine learning in wildlife research

Camera trapping has become an important tool in wildlife research in the past few decades. However, one of its main limiting factors is the processing of data, which is labour-intensive and time-consuming. Consequently, to aid this process, the use of machine learning has increased. A summary is provided on the use of both camera traps and machine learning and the main challenges that come with it by performing a general literature review. Remote cameras can be used in a variety of field applications, including investigating species distribution, disease transmission and vaccination, population estimation, nest predation, animal activity patterns, wildlife crossings, and diet analysis. Camera trapping has many benefits, including being less invasive, allowing for consistent monitoring and simultaneous observation (especially of secretive or aggressive animals even in dangerous or remote areas), providing photo/video evidence, reducing observer bias, and being cost effective. The main issues are that they are subject to their environment, dependent on human placements, can disrupt animal behaviour, need maintenance and repair, have limitations on photographic data, and are sensitive to theft and vandalism. When it comes to machine learning, the main aim is to identify species in camera (trap) images, although emerging technologies can provide individual recognition as well. The downsides include the large amount of annotated data, computer power, and programming and machine learning expertise needed. Nonetheless, camera trapping and machine learning can greatly assist ecologists and conservationists in wildlife research, even more so as technology further develops

AN EVALUATION OF CITIZEN SCIENCE-BASED INDICES FOR MONITORING THE DISTRIBUTION AND ABUNDANCE OF BOBCATS (LYNX RUFUS)

Carnivores substantially impact humans, but are elusive and difficult to monitor. Although less precise than intensive methods (e.g., capture-recapture), indices of relative abundance are widely used to monitor carnivore numbers. This study assessed public sightings and hunter surveys as approaches to monitoring the distribution and relative abundance of bobcats (Lynx rufus) in New Hampshire. To validate indices, I used a telemetry-based model of habitat suitability and information from camera surveys conducted by volunteers in three study areas. Bobcats were found widely distributed in New Hampshire with lower abundance in northern and mountainous regions. Public sightings and hunter surveys (both effort-corrected) were strongly correlated to each other and the suitability model when summarized by Wildlife Management Unit. Detection rates from camera surveys were correlated to other indices and the model within the three study areas. I suggest future research validate indices using absolute abundance, and assess influences of confounding variables