29 research outputs found
Deep Learning Pipeline for Automated Visual Moth Monitoring: Insect Localization and Species Classification
Biodiversity monitoring is crucial for tracking and counteracting adverse
trends in population fluctuations. However, automatic recognition systems are
rarely applied so far, and experts evaluate the generated data masses manually.
Especially the support of deep learning methods for visual monitoring is not
yet established in biodiversity research, compared to other areas like
advertising or entertainment. In this paper, we present a deep learning
pipeline for analyzing images captured by a moth scanner, an automated visual
monitoring system of moth species developed within the AMMOD project. We first
localize individuals with a moth detector and afterward determine the species
of detected insects with a classifier. Our detector achieves up to 99.01% mean
average precision and our classifier distinguishes 200 moth species with an
accuracy of 93.13% on image cutouts depicting single insects. Combining both in
our pipeline improves the accuracy for species identification in images of the
moth scanner from 79.62% to 88.05%
End-to-end Learning of a Fisher Vector Encoding for Part Features in Fine-grained Recognition
Part-based approaches for fine-grained recognition do not show the expected
performance gain over global methods, although being able to explicitly focus
on small details that are relevant for distinguishing highly similar classes.
We assume that part-based methods suffer from a missing representation of local
features, which is invariant to the order of parts and can handle a varying
number of visible parts appropriately. The order of parts is artificial and
often only given by ground-truth annotations, whereas viewpoint variations and
occlusions result in parts that are not observable. Therefore, we propose
integrating a Fisher vector encoding of part features into convolutional neural
networks. The parameters for this encoding are estimated jointly with those of
the neural network in an end-to-end manner. Our approach improves
state-of-the-art accuracies for bird species classification on CUB-200-2011
from 90.40\% to 90.95\%, on NA-Birds from 89.20\% to 90.30\%, and on Birdsnap
from 84.30\% to 86.97\%
Classification-Specific Parts for Improving Fine-Grained Visual Categorization
Fine-grained visual categorization is a classification task for
distinguishing categories with high intra-class and small inter-class variance.
While global approaches aim at using the whole image for performing the
classification, part-based solutions gather additional local information in
terms of attentions or parts. We propose a novel classification-specific part
estimation that uses an initial prediction as well as back-propagation of
feature importance via gradient computations in order to estimate relevant
image regions. The subsequently detected parts are then not only selected by
a-posteriori classification knowledge, but also have an intrinsic spatial
extent that is determined automatically. This is in contrast to most part-based
approaches and even to available ground-truth part annotations, which only
provide point coordinates and no additional scale information. We show in our
experiments on various widely-used fine-grained datasets the effectiveness of
the mentioned part selection method in conjunction with the extracted part
features.Comment: Presented at the GCPR201
Automated Visual Monitoring of Nocturnal Insects with Light-based Camera Traps
Automatic camera-assisted monitoring of insects for abundance estimations is
crucial to understand and counteract ongoing insect decline. In this paper, we
present two datasets of nocturnal insects, especially moths as a subset of
Lepidoptera, photographed in Central Europe. One of the datasets, the EU-Moths
dataset, was captured manually by citizen scientists and contains species
annotations for 200 different species and bounding box annotations for those.
We used this dataset to develop and evaluate a two-stage pipeline for insect
detection and moth species classification in previous work. We further
introduce a prototype for an automated visual monitoring system. This prototype
produced the second dataset consisting of more than 27,000 images captured on
95 nights. For evaluation and bootstrapping purposes, we annotated a subset of
the images with bounding boxes enframing nocturnal insects. Finally, we present
first detection and classification baselines for these datasets and encourage
other scientists to use this publicly available data.Comment: Presented at the FGVC workshop at the CVPR202
Maximally Divergent Intervals for Anomaly Detection
We present new methods for batch anomaly detection in multivariate time
series. Our methods are based on maximizing the Kullback-Leibler divergence
between the data distribution within and outside an interval of the time
series. An empirical analysis shows the benefits of our algorithms compared to
methods that treat each time step independently from each other without
optimizing with respect to all possible intervals.Comment: ICML Workshop on Anomaly Detectio
Pre-trained models are not enough: active and lifelong learning is important for long-term visual monitoring of mammals in biodiversity research—Individual identification and attribute prediction with image features from deep neural networks and decoupled decision models applied to elephants and great apes
Animal re-identification based on image data, either recorded manually by photographers or automatically with camera traps, is an important task for ecological studies about biodiversity and conservation that can be highly automatized with algorithms from computer vision and machine learning. However, fixed identification models only trained with standard datasets before their application will quickly reach their limits, especially for long-term monitoring with changing environmental conditions, varying visual appearances of individuals over time that differ a lot from those in the training data, and new occurring individuals that have not been observed before. Hence, we believe that active learning with human-in-the-loop and continuous lifelong learning is important to tackle these challenges and to obtain high-performance recognition systems when dealing with huge amounts of additional data that become available during the application. Our general approach with image features from deep neural networks and decoupled decision models can be applied to many different mammalian species and is perfectly suited for continuous improvements of the recognition systems via lifelong learning. In our identification experiments, we consider four different taxa, namely two elephant species: African forest elephants and Asian elephants, as well as two species of great apes: gorillas and chimpanzees. Going beyond classical re-identification, our decoupled approach can also be used for predicting attributes of individuals such as gender or age using classification or regression methods. Although applicable for small datasets of individuals as well, we argue that even better recognition performance will be achieved by improving decision models gradually via lifelong learning to exploit huge datasets and continuous recordings from long-term applications. We highlight that algorithms for deploying lifelong learning in real observational studies exist and are ready for use. Hence, lifelong learning might become a valuable concept that supports practitioners when analyzing large-scale image data during long-term monitoring of mammals
Earth system data cubes unravel global multivariate dynamics
Understanding Earth system dynamics in light of ongoing human intervention and dependency remains a major scientific challenge. The unprecedented availability of data streams describing different facets of the Earth now offers fundamentally new avenues to address this quest. However, several practical hurdles, especially the lack of data interoperability, limit the joint potential of these data streams. Today, many initiatives within and beyond the Earth system sciences are exploring new approaches to overcome these hurdles and meet the growing interdisciplinary need for data-intensive research; using data cubes is one promising avenue. Here, we introduce the concept of Earth system data cubes and how to operate on them in a formal way. The idea is that treating multiple data dimensions, such as spatial, temporal, variable, frequency, and other grids alike, allows effective application of user-defined functions to co-interpret Earth observations and/or model-data integration. An implementation of this concept combines analysis-ready data cubes with a suitable analytic interface. In three case studies, we demonstrate how the concept and its implementation facilitate the execution of complex workflows for research across multiple variables, and spatial and temporal scales: (1) summary statistics for ecosystem and climate dynamics; (2) intrinsic dimensionality analysis on multiple timescales; and (3) model-data integration. We discuss the emerging perspectives for investigating global interacting and coupled phenomena in observed or simulated data. In particular, we see many emerging perspectives of this approach for interpreting large-scale model ensembles. The latest developments in machine learning, causal inference, and model-data integration can be seamlessly implemented in the proposed framework, supporting rapid progress in data-intensive research across disciplinary boundaries. © 2020 Institute of Electrical and Electronics Engineers Inc.. All rights reserved