Deep learning for species identification of modern and fossil rodent molars

Reliable identification of species is a key step to assess biodiversity. In fossil and archaeological contexts, genetic identifications remain often difficult or even impossible and morphological criteria are the only window on past biodiversity. Methods of numerical taxonomy based on geometric morphometric provide reliable identifications at the specific and even intraspecific levels, but they remain relatively time consuming and require expertise on the group under study. Here, we explore an alternative based on computer vision and machine learning. The identification of three rodent species based on pictures of their molar tooth row constituted the case study. We focused on the first upper molar in order to transfer the model elaborated on modern, genetically identified specimens to isolated fossil teeth. A pipeline based on deep neural network automatically cropped the first molar from the pictures, and returned a prediction regarding species identification. The deep-learning approach performed equally good as geometric morphometrics and, provided an extensive reference dataset including fossil teeth, it was able to successfully identify teeth from an archaeological deposit that was not included in the training dataset. This is a proof-of-concept that such methods could allow fast and reliable identification of extensive amounts of fossil remains, often left unstudied in archaeological deposits for lack of time and expertise. Deep-learning methods may thus allow new insights on the biodiversity dynamics across the last 10.000 years, including the role of humans in extinction or recent evolution

Learning to segment prostate cancer by aggressiveness from scribbles in bi-parametric MRI

International audienceIn this work, we propose a deep U-Net based model to tackle the challenging task of prostate cancer segmentation by aggressiveness in MRI based on weak scribble annotations. This model extends the size constraint loss proposed by Kervadec et al. 1 in the context of multiclass detection and segmentation task. This model is of high clinical interest as it allows training on prostate biopsy samples and avoids time-consuming full annotation process. Performance is assessed on a private dataset (219 patients) where the full ground truth is available as well as on the ProstateX-2 challenge database, where only biopsy results at different localisations serve as reference. We show that we can approach the fully-supervised baseline in grading the lesions by using only 6.35% of voxels for training. We report a lesion-wise Cohen's kappa score of 0.29 ± 0.07 for the weak model versus 0.32 ± 0.05 for the baseline. We also report a kappa score (0.276 ± 0.037) on the ProstateX-2 challenge dataset with our weak U-Net trained on a combination of ProstateX-2 and our dataset, which is the highest reported value on this challenge dataset for a segmentation task to our knowledge

Perfusion imaging in deep prostate cancer detection from mp-MRI: can we take advantage of it?

International audienceTo our knowledge, all deep computer-aided detection and diagnosis (CAD) systems for prostate cancer (PCa) detection consider bi-parametric magnetic resonance imaging (bp-MRI) only, including T2w and ADC sequences while excluding the 4D perfusion sequence,which is however part of standard clinical protocols for this diagnostic task. In this paper, we question strategies to integrate information from perfusion imaging in deep neural architectures. To do so, we evaluate several ways to encode the perfusion information in a U-Net like architecture, also considering early versus mid fusion strategies. We compare performance of multiparametric MRI (mp-MRI) models with the baseline bp-MRI model based on a private dataset of 219 mp-MRI exams. Perfusion maps derived from dynamic contrast enhanced MR exams are shown to positively impact segmentation and grading performance of PCa lesions, especially the 3D MR volume corresponding to the maximum slope of the wash-in curve as well as Tmax perfusion maps. The latter mp-MRI models indeed outperform the bp-MRI one whatever the fusion strategy, with Cohen's kappa score of 0.318±0.019 for the bp-MRI model and 0.378 ± 0.033 for the model including the maximum slope with a mid fusion strategy, also achieving competitive Cohen's kappa score compared to state of the art

Beyond accuracy : score calibration in deep learning models for camera trap image sequences

In this paper, we investigate whether deep learning models for species classification in camera trap images are well calibrated, i.e. whether predicted confidence scores can be reliably interpreted as probabilities that the predictions are true. Additionally, as camera traps are often configured to take multiple photos of the same event, we also explore the calibration of predictions at the sequence level. Here, we (i) train deep learning models on a large and diverse European camera trap dataset, using five established architectures; (ii) compare their calibration and classification performances on three independent test sets; (iii) measure the performances at sequence level using four approaches to aggregate individuals predictions; (iv) study the effect and the practicality of a post-hoc calibration method, for both image and sequence levels. Our results first suggest that calibration and accuracy are closely intertwined and vary greatly across model architectures. Secondly, we observe that averaging the logits over the sequence before applying softmax normalization emerges as the most effective method for achieving both good calibration and accuracy at the sequence level. Finally, temperature scaling can be a practical solution to further improve calibration, given the generalizability of the optimum temperature across datasets. We conclude that, with adequate methodology, deep learning models for species classification can be very well calibrated. This considerably improves the interpretability of the confidence scores and their usability in ecological downstream tasks

Revisiting animal photo-identification using deep metric learning and network analysis

An increasing number of ecological monitoring programmes rely on photographic capture–recapture of individuals to study distribution, demography and abundance of species. Photo-identification of individuals can sometimes be done using idiosyncratic coat or skin patterns, instead of using tags or loggers. However, when performed manually, the task of going through photographs is tedious and rapidly becomes too time-consuming as the number of pictures grows. Computer vision techniques are an appealing and unavoidable help to tackle this apparently simple task in the big-data era. In this context, we propose to revisit animal re-identification using image similarity networks and metric learning with convolutional neural networks (CNNs), taking the giraffe as a working example. We first developed an end-to-end pipeline to retrieve a comprehensive set of re-identified giraffes from about 4,000 raw photographs. To do so, we combined CNN-based object detection, SIFT pattern matching and image similarity networks. We then quantified the performance of deep metric learning to retrieve the identity of known individuals, and to detect unknown individuals never seen in the previous years of monitoring. After a data augmentation procedure, the re-identification performance of the CNN reached a Top-1 accuracy of about 90%, despite the very small number of images per individual in the training dataset. While the complete pipeline succeeded in re-identifying known individuals, it slightly under-performed with unknown individuals. Fully based on open-source software packages, our work paves the way for further attempts to build automatic pipelines for re-identification of individual animals, not only in giraffes but also in other species.DATA AVAILABILITY STATEMENT: The curated dataset of re-identified giraffe individuals is freely available at ftp://pbil.univ-lyon1.fr/pub/datasets/miele2021. The code to reproduce the analysis is available at https://plmlab.math.cnrs.fr/vmiele/animal-reid/ with explanations and test cases.French National Center for Scientific Research (CNRS) and Statistical Ecology Research Group (EcoStat).https://besjournals.onlinelibrary.wiley.com/journal/2041210x2022-03-17hj2021Mammal Research InstituteZoology and Entomolog

ProstAttention-Net: a deep attention model for prostate cancer segmentation by aggressiveness in MRI scans

International audienceMultiparametric magnetic resonance imaging (mp-MRI) has shown excellent results in the detection of prostate cancer (PCa). However, characterizing prostate lesions aggressiveness in mp-MRI sequences is impossible in clinical practice, and biopsy remains the reference to determine the Gleason score (GS). In this work, we propose a novel end-to-end multi-class network that jointly segments the prostate gland and cancer lesions with GS group grading. After encoding the information on a latent space, the network is separated in two branches: 1) the first branch performs prostate segmentation 2) the second branch uses this zonal prior as an attention gate for the detection and grading of prostate lesions. The model was trained and validated with a 5-fold cross-validation on an heterogeneous series of 219 MRI exams acquired on three different scanners prior prostatectomy. In the free-response receiver operating characteristics (FROC) analysis for clinically significant lesions (defined as GS > 6) detection, our model achieves 69.0% ±14.5% sensitivity at 2.9 false positive per patient on the whole prostate and 70.8% ±14.4% sensitivity at 1.5 false positive when considering the peripheral zone (PZ) only. Regarding the automatic GS grou

Revisiting giraffe photo-identification using deep learning and network analysis

An increasing number of research programs rely on photographic capture-recapture (vs. direct marking) of individuals to study distribution and demography within animal populations. Photo-identification of individuals living in the wild is sometimes feasible using idiosyncratic coat or skin patterns, like for giraffes. When performed manually, the task is tedious and becomes almost impossible as populations grow in size. Computer vision techniques are an appealing and unavoidable help to tackle this apparently simple task in the big-data era. In this context, we propose to revisit giraffe re-identification using convolutional neural networks (CNNs).We first developed an end-to-end pipeline to retrieve a comprehensive set of re-identified giraffes from about 4, 000 raw photographs. To do so, we combined CNN-based object detection, SIFT pattern matching, and image similarity networks. We then quantified the performance of deep metric learning to retrieve the identity of known and unknown individuals. The re-identification performance of CNNs reached a top 5 accuracy of about 90%. Fully based on open-source software packages, our work paves the way for further attempts to build CNN-based pipelines for re-identification of individual animals, in giraffes but also in other species

The DeepFaune initiative: a collaborative effort towards the automatic identification of Europeanfauna in camera trap images

Camera traps have revolutionized how ecologists monitor wildlife, but their full potential is realized only when the hundreds of thousands of collected images can be readily classified with minimal human intervention. Deep-learning classification models have allowed extraordinary progress towards this end, but trained models remain rare and are only now emerging for European fauna. We report on the first milestone of the DeepFaune initiative (https://www.deepfaune.cnrs.fr), a large-scale collaboration between more than 50 partners involved in wildlife research, conservation and management in France. We developed aclassification model trained to recognize 26 species or higher-level taxa that are common in Europe, with an emphasis on mammals. The classification model achieved 0.97 validation accuracy and often >0.95 precision and recall for many classes. These performances were generally higher than 0.90 when tested on independent out-of-sample datasets for which we used image redundancy contained in sequences of images. We implemented our model in a software to classify images stored locally on a personal computer, so as to provide a free, user-friendly and high-performance tool for wildlife practitioners to automatically classify camera trap images. The DeepFaune initiative is an ongoing project, with new partners joining regularly,which allows us to continuously add new species to the classification model