Integrating Visual and Semantic Similarity Using Hierarchies for Image Retrieval

Most of the research in content-based image retrieval (CBIR) focus on developing robust feature representations that can effectively retrieve instances from a database of images that are visually similar to a query. However, the retrieved images sometimes contain results that are not semantically related to the query. To address this, we propose a method for CBIR that captures both visual and semantic similarity using a visual hierarchy. The hierarchy is constructed by merging classes with overlapping features in the latent space of a deep neural network trained for classification, assuming that overlapping classes share high visual and semantic similarities. Finally, the constructed hierarchy is integrated into the distance calculation metric for similarity search. Experiments on standard datasets: CUB-200-2011 and CIFAR100, and a real-life use case using diatom microscopy images show that our method achieves superior performance compared to the existing methods on image retrieval.Comment: Accepted in ICVS 202

Self-Supervised Gaussian Regularization of Deep Classifiers for Mahalanobis-Distance-Based Uncertainty Estimation

Recent works show that the data distribution in a network's latent space is useful for estimating classification uncertainty and detecting Out-of-distribution (OOD) samples. To obtain a well-regularized latent space that is conducive for uncertainty estimation, existing methods bring in significant changes to model architectures and training procedures. In this paper, we present a lightweight, fast, and high-performance regularization method for Mahalanobis distance-based uncertainty prediction, and that requires minimal changes to the network's architecture. To derive Gaussian latent representation favourable for Mahalanobis Distance calculation, we introduce a self-supervised representation learning method that separates in-class representations into multiple Gaussians. Classes with non-Gaussian representations are automatically identified and dynamically clustered into multiple new classes that are approximately Gaussian. Evaluation on standard OOD benchmarks shows that our method achieves state-of-the-art results on OOD detection with minimal inference time, and is very competitive on predictive probability calibration. Finally, we show the applicability of our method to a real-life computer vision use case on microorganism classification.Comment: 24 pages including supplementary materia

In vivo estimation of pigment composition and optical absorption cross-sectionby spectroradiometry in four aquatic photosynthetic micro-organisms

International audienceThe objective of the present study was to estimate in vivo pigment composition and to retrieve absorption cross-section values, a∗, of photosynthetic micro-organisms using a non-invasive technique of reflectance spectrometry. To test the methodology, organisms from different taxonomical groups and different pigment composition were used (Spirulina platensis a Cyanophyta, Porphyridium cruentum a Rhodophyta, Dunaliella tertiolecta a Chlorophyta and Entomoneis paludosa a Bacillariophyta) and photoacclimated to two different irradiance levels: 25 μmol photon m−2 s−1 (Low Light, LL) and 500 μmol photon m−2 s−1 (High Light, HL). Second derivative spectra from reflectance were used to identify pigment in vivo absorption bands that were linked to specific pigments detected by high performance liquid chromatography. Whereas some absorption bands such as those induced by Chlorophyll (Chl) a (416, 440, 625 and around 675 nm) were ubiquous, others were taxonomically specific (e.g. 636 nm for Chl c in E. paludosa) and/or photo-physiological dependent (e.g. 489 nm for zeaxanthin in the HL-acclimated S. platensis). The optical absorption cross-section, a∗, was retrieved from reflectance data using a radiative transfer model previously developed for microphytobenthos. Despite the cellular Chl a decrease observed from LL to HL (up to 88% for S. platensis), the a∗ increased, except for P. cruentum. This was attributed to a ‘package effect’ and to a greater absorption by photoprotective carotenoids that did not contribute to the energy transfer to the core Chl a

Temperature and nutrient effects on the relative importance of brown and green pathways for stream ecosystem functioning: A mesocosm approach

In addition to global warming, aquatic ecosystems are currently facing multiple global changes among which include changes in nitrogen (N) loads. While several studies have investigated both temperature and N impacts on aquatic ecosystems independently, knowledge on their interactive effects remains scarce. In forested headwater streams, decomposition of leaf litter represents the main process ensuring the transfer of nutrients and energy to higher trophic levels, followed by autochthonous primary production, mainly ensured by phototrophic biofilms. The main aim of this study was to disentangle the independent and combined effects of temperature increase and nutrient availability on the relative importance of brown and green processes involved in stream functioning. We hypothesised that water temperature and nutrients would lead to a general increase in leaf‐litter decomposition and primary production, but that the intensity of these effects would be largely modulated by competitive interactions arising between microorganisms as well as by the top‐down control of microorganisms by macro‐invertebrates. Macro‐invertebrates would, in turn, be bottom‐up controlled by microbial resources quality. To test these hypotheses, we conducted a 56‐day experiment in artificial streams containing leaf litter, microbial decomposers and biofilm inoculum, and an assemblage of macro‐invertebrates. Two water inorganic N:phosphorus (P) ratios (33 and 100, molar ratios) and two temperatures (ambient, +2°C) were manipulated, each treatment being replicated three times. Fungal and biofilm growth as well as leaf‐litter decomposition and primary production were quantified. Top‐down impacts of invertebrate primary consumers on brown and green compartments were evaluated using exclosures while bottom‐up control was evaluated through the measurement of resource stoichiometry and fatty acid profiles, as well as quantification of macro‐invertebrate growth and survival. Contrary to expectations, microbial decomposition was not significantly stimulated by nutrient or temperature manipulations, while primary production was only improved under ambient temperature. In the + 2°C treatment with high N:P, greater biofilm biomass was associated with lower fungal development, which indicates competition for nutrients in these conditions. Temperature increased macro‐invertebrate growth and leaf‐litter consumption, but this effect was independent of any improvement of basal resource quality, suggesting that temperature mediated changes in consumer metabolism and activity was the main mechanism involved. Most of our hypotheses that were based on simplified laboratory observations have been rejected in our semi‐controlled mesocosms. Our study suggests that the complexity of biological communities might greatly affect the response of ecosystems to multiple stressors, and that interactions between organisms must be explicitly taken into account when investigating the impacts of global change on ecosystem functioning

Machine learning techniques to characterize functional traits of plankton from image data

Plankton imaging systems supported by automated classification and analysis have improved ecologists' ability to observe aquatic ecosystems. Today, we are on the cusp of reliably tracking plankton populations with a suite of lab-based and in situ tools, collecting imaging data at unprecedentedly fine spatial and temporal scales. But these data have potential well beyond examining the abundances of different taxa; the individual images themselves contain a wealth of information on functional traits. Here, we outline traits that could be measured from image data, suggest machine learning and computer vision approaches to extract functional trait information from the images, and discuss promising avenues for novel studies. The approaches we discuss are data agnostic and are broadly applicable to imagery of other aquatic or terrestrial organisms

Reconnaissance automatique des diatomées par ordinateur : un défi toujours d’actualité

International audienceL’identification des diatomées au microscope est une tâche ardue, chronophage et sujette à de multiples biais (expérience de l'opérateur, qualité de l'image). Les premières tentatives de classification automatique de ces organismes datent des années 90, mais le développement d’une approche robuste constitue toujours un défi d’actualité. Le développement récent d’une nouvelle génération de modèles mathématiques dit d’apprentissage profond ou deep learning semble prometteur pour résoudre les problèmes rencontrés jusqu’à présent Cette présentation aura tout d’abord pour objectif d’introduire les grands principes de cette nouvelle approche pour l’analyse automatique des images par ordinateur. Dans un second temps, son apport potentiel pour l’écologie sera illustré, en particulier dans le cadre de projets en cours portant sur les diatomées, que cela soit dans le cadre du diagnostic écologique des cours d’eau, la biométrie ou encore l’apparition de formes rares (e.g., espèces exotiques, déformations…). Enfin les principaux obstacles rencontrés dans la construction de l’outil et ses limites seront abordés

Monitoring of pollutant effects on natural periphytic communities using chlorophyll fluorescence measurements

Dans le cadre de l'application de la Directive Cadre sur l'Eau il serait intéressant de mettre au point de nouveaux outils de surveillance de la qualité des cours d'eau. Dans le cas d'une pollution ponctuelle, certains d'entre eux pourraient être basés sur l'évaluation de l'état physiologique des communautés de diatomées périphytiques qui constituent souvent le premier maillon des écosystèmes aquatiques continentaux. L'objectif de ce travail était d'étudier l'effet de polluants communément retrouvés dans les milieux aquatiques (Cu, Zn, Cd, atrazine, isoproturon) sur la réponse de communautés périphytiques naturelles et de mieux cerner l'influence de la lumière. Nous avons mis au point une démarche méthodologique originale, basée sur l'estimation de leur activité photosynthétique à l'aide de la technique de mesure de la fluorescence chlorophyllienne en lumière modulée (fluorescence PAM). Nos résultats ont mis en évidence une certaine résistance à court terme des communautés aux différents polluants testés, variable selon le mode d'action du polluant (herbicides vs métaux lourds) et la structure du biofilm (biomasse, composition spécifique). En combinant des expérimentations originales en conditions naturelles et au laboratoire, nous avons montré que l'application d'un stress supplémentaire comme la lumière amplifie de manière significative la toxicité de ces contaminants. Ce travail confirme tout l'intérêt de l'utilisation de la technique de mesure de la fluorescence chlorophyllienne en lumière modulée dans le cadre de bioessais écotoxicologiques. Il offre également des perspectives intéressantes en vue de la détection in situ de pollutions ponctuelles par les herbicides.Ln the context of the European Water Framework Directive implementation, new tools for river toxicity assessment have to be developed. Periphytic communities, which play a fundamental role in the trophic web in lotic systems, could be regarded as early wamings for detection of acute toxicant exposure by monitoring its physiological state. Several studies were then carried out by means of the chlorophyll fluorescence measurement in modulated light technique (P AM fluorescence) which helps to estimate the photosynthetic activity in a non-intrusive way. The aim of the present work was to study the effects of several pollutants commonly found in aquatic ecosystems (Cu, Zn, Cd, atrazine, isoproturon) on stream periphyton physiology and to highlight the role of light as an addition al potentially stressful parameter. Our results showed a relative resistance of the natural biofilms to the pollutants on a short term scale (< 1 day) which depended both on the mechanism of action of the toxicant (herbicides vs heavy metals) and the biofilms architecture. By combining novel experimental designs both in field and laboratory conditions, we also demonstrated that periphyton can be more affected when another stress, such as light, is applied. This work confmns the usefulness of chlorophyll fluorescence-based methods in ecotoxicological studies, particularly in order to detect in situ herbicide toxic effects

Regards croisés: Using artificial intelligence for ecology, perspectives from a computer scientist and an ecologist

International audienceMachine learning is now well established as a powerful tool for solving ecological questions. Yet, it is not a trivial task to develop an interdisciplinary approach involving experts of very different scientific fields, but also students at the interface. From our experience, the likelihood of developing long-term collaboration increases with the ability of experts to develop a common language in order to understand each other. To illustrate this, we will discuss several keywords, such as “interdisciplinarity”, “methodology” and “explainability”, which can hold different meanings depending on the perspective of a computer scientist or an ecologist. This discussion will be open to the audience and serve as a conclusion of the symposium "Machine learning for ecological images

Regards croisés: Using artificial intelligence for ecology, perspectives from a computer scientist and an ecologist

International audienceMachine learning is now well established as a powerful tool for solving ecological questions. Yet, it is not a trivial task to develop an interdisciplinary approach involving experts of very different scientific fields, but also students at the interface. From our experience, the likelihood of developing long-term collaboration increases with the ability of experts to develop a common language in order to understand each other. To illustrate this, we will discuss several keywords, such as “interdisciplinarity”, “methodology” and “explainability”, which can hold different meanings depending on the perspective of a computer scientist or an ecologist. This discussion will be open to the audience and serve as a conclusion of the symposium "Machine learning for ecological images