Pollen grain recognition through deep learning convolutional neural networks

Palynology is the study of pollen, in particular, the pollen’s grain type, but the tasks of classification and counting of pollen grains are highly skilled and laborious. Despite the efforts made during the last decades, the manual classification process is still predominant. One of the reasons for that is the small number of taxa usually used in previous approaches. In this paper, we propose a new method to automatically classify pollen grains using a state-of-the-art deep learning technique applied to the recently published POLEN73S image dataset. Our proposal manages to classify up to 94% of the samples from the dataset with 73 different classes of pollen grains. This result, which surpasses all previous attempts in number and difficulty of taxa under consideration, gives good perspectives to achieve a perfect score in pollen recognition task even with a large number of pollen grain types.info:eu-repo/semantics/publishedVersio

Spatially Aware Dictionary Learning and Coding for Fossil Pollen Identification

We propose a robust approach for performing automatic species-level recognition of fossil pollen grains in microscopy images that exploits both global shape and local texture characteristics in a patch-based matching methodology. We introduce a novel criteria for selecting meaningful and discriminative exemplar patches. We optimize this function during training using a greedy submodular function optimization framework that gives a near-optimal solution with bounded approximation error. We use these selected exemplars as a dictionary basis and propose a spatially-aware sparse coding method to match testing images for identification while maintaining global shape correspondence. To accelerate the coding process for fast matching, we introduce a relaxed form that uses spatially-aware soft-thresholding during coding. Finally, we carry out an experimental study that demonstrates the effectiveness and efficiency of our exemplar selection and classification mechanisms, achieving $86.13\%$ accuracy on a difficult fine-grained species classification task distinguishing three types of fossil spruce pollen.Comment: CVMI 201

Automatic pollen recognition using convolutional neural networks: The case of the main pollens present in Spanish citrus and rosemary honey

[EN] The automation of honey pollen visual sorting overcomes the limitations of the conventional procedure helping the specialist in this time-consuming task. In this work, a novel and comprehensive Ground Truth of almost 19,000 images (from optical microscopy) of the 16 most abundant types of grains/pollen particles present in citrus and rosemary honey from Spain was constructed. This task was assisted by a HoneyApp (also developed herein) for the labelling and annotation process. Subsequently, the effectiveness of different pre-existing automatic pollen recognizers based on convolutional neural networks (CNN) (VGG16, VGG19, InceptionV3, Xception, ResNet50, DenseNet201, MobileNetV2 and EfficientNetV2M) was tested together with a new network proposed in this paper (PolleNetV1). The extreme complexity of those pre-existing CNN and extensive use of millions of parameters makes this new proposal especially promising. Although with a slightly lower accuracy (average 96%) in determining the relative frequencies of different types of pollen grains/particles, it has considerable advantages such as simplicity and ability to be included in the future functionality to automate pollen recognition in honey. This is the first step to finally achieving an objective tool that allows the correct labelling of any types of pollen in honey, thus contributing to its transparency in the market.This work is part of Spanish project PID2019-106800RB-I00 (2019) with financial support from the Ministerio de Ciencia e Innovacion (MCIN), Agencia Estatal de Investigacion MCIN/AEI/10.13039/501100011033/. It has been also part of the project AGROALNEXT/2022/043, funded by the Next Generation European Union and the Plan de Recuperacion, Transformacion y Resiliencia of the Spanish Government, with the support of Generalitat Valenciana. The authors would like to thank the CRUE-Universitat Politecnica deValencia for providing the funds for open access publication.Valiente González, JM.; Juan-Borras, MDS.; López García, F.; Escriche Roberto, MI. (2023). Automatic pollen recognition using convolutional neural networks: The case of the main pollens present in Spanish citrus and rosemary honey. Journal of Food Composition and Analysis. 123:1-10. https://doi.org/10.1016/j.jfca.2023.10560511012

Addressing subjectivity in the classification of palaeoenvironmental remains with supervised deep learning convolutional neural networks

Archaeological object identifications have been traditionally undertaken through a comparative methodology where each artefact is identified through a subjective, interpretative act by a professional. Regarding palaeoenvironmental remains, this comparative methodology is given boundaries by using reference materials and codified sets of rules, but subjectivity is nevertheless present. The problem with this traditional archaeological methodology is that higher level of subjectivity in the identification of artefacts leads to inaccuracies, which then increases the potential for Type I and Type II errors in the testing of hypotheses. Reducing the subjectivity of archaeological identifications would improve the statistical power of archaeological analyses, which would subsequently lead to more impactful research. In this thesis, it is shown that the level of subjectivity in palaeoenvironmental research can be reduced by applying deep learning convolutional neural networks within an image recognition framework. The primary aim of the presented research is therefore to further the on-going paradigm shift in archaeology towards model-based object identifications, particularly within the realm of palaeoenvironmental remains. Although this thesis focuses on the identification of pollen grains and animal bones, with the latter being restricted to the astragalus of sheep and goats, there are wider implications for archaeology as these methods can easily be extended beyond pollen and animal remains. The previously published POLEN23E dataset is used as the pilot study of applying deep learning in pollen grain classification. In contrast, an image dataset of modern bones was compiled for the classification of sheep and goat astragali due to a complete lack of available bone image datasets and a double blind study with inexperienced and experienced zooarchaeologists was performed to have a benchmark to which image recognition models can be compared. In both classification tasks, the presented models outperform all previous formal modelling methods and only the best human analysts match the performance of the deep learning model in the sheep and goat astragalus separation task. Throughout the thesis, there is a specific focus on increasing trust in the models through the visualization of the models’ decision making and avenues of improvements to Grad-CAM are explored. This thesis makes an explicit case for the phasing out of the comparative methods in favour of a formal modelling framework within archaeology, especially in palaeoenvironmental object identification

Unsupervised Representations of Pollen in Bright-Field Microscopy

We present the first unsupervised deep learning method for pollen analysis using bright-field microscopy. Using a modest dataset of 650 images of pollen grains collected from honey, we achieve family level identification of pollen. We embed images of pollen grains into a low-dimensional latent space and compare Euclidean and Riemannian metrics on these spaces for clustering. We propose this system for automated analysis of pollen and other microscopic biological structures which have only small or unlabelled datasets available.Comment: Accepted at the Workshop on Computational Biology at the International Conference on Machine Learning (ICML) in Long Beach, CA, USA on June 14, 201

New methods in Palaeopalynology: Classification of pollen through pollen chemistry

Pollen grains are one of the primary tools of palaeoecologists to reconstruct vegetation changes in the past. The description, counting and analysis of pollen grains (palynology) has contributed to our understanding of establishment and dynamics of past and present plant communities. Advances in identification accuracy, precision and increased taxonomic resolution have greatly improved our understanding of biogeography and plant community interactions. Nevertheless, the techniques by which palynological studies are performed have not fundamentally changed. Taxonomic resolution and automation have been identified as some of the key challenges for palynology and palaeoecology. Chemical methods have been proposed as a potential alternative to morphological approaches and have demonstrated promising results in the classification of modern pollen grains and in the analysis of pollen chemical responses to UV-B radiation. The application of chemical methods for palynological needs have not been thoroughly explored, with analysis of (sub-)fossil pollen lagging behind their modern counterpart. Especially the application of infrared methods have gained popularity as an alternative to traditional morphological approaches. In this thesis, I explore the use of infrared methods for palynological applications, by exploring the chemical variation in modern pollen grains and in the analysis of fossil pollen grains with IR microscope approaches. The objectives of this thesis are formulated into three research objectives: * Collect modern pollen and explore the variation in chemical composition * Apply chemical methods to fossil material * Explore microscopy chemical methods on modern pollen The thesis is structured into four studies to study these objectives. Papers I and II explore variation and classification based on the chemical composition of modern *Quercus* pollen using two IR approaches, Fourier transform infrared spectroscopy (FTIR) and Fourier transform Raman spectroscopy (FT-Raman). After exploring modern chemical composition of pollen, paper III investigates FTIR methods for the analysis of fossil pollen, in spectra of Holocene *Pinus* pollen. Additionally, the effects of acetolysis and density separation on *Pinus* pollen is described. Paper IV addresses the challenge of scattering signals when measuring small pollen grains of four *Quercus* species with FTIR microscopy and ways to surpress or weaken the scattering signals. The results from paper I and II show classification success, surpassing traditional morphological approaches, at the *Quercus* section level and ~90% recall on species level with both IR approaches. Chemical bands most useful for classification are lipids, sporopollenin and proteins for both FT-Raman and FTIR. We observe differences in the importance of chemical functional groups for the classification. FT-Raman relies more on sporopollenin chemistry, while FTIR utilizes more variation in lipid bands. After finding considerable variation in sporopollenin chemistry in modern pollen samples, FTIR methods were applied to pollen from sediment cores spanning the Holocene. Paper III examines the differences between modern and sub-fossil pollen and reported large differences between them, mainly the removal of labile components, such as lipids and protein peaks from the sub-fossil spectra during diagenesis. Additionally, paper III finds changes to pollen chemistry caused by acetolysis in the 1200 - 1000 cm^-1^ region of the spectra, when comparing acetolysed spectra to non-acetolysed spectra. The paper concludes with findings of unwanted inorganic signals (BSi) and contamination from density separation media in the sediment pollen spectra. Paper IV demonstrates two successful methods of removing scattering signals from pollen spectra. Two approaches were examined, embedding and processing with signal correction algorithms. Spectra from embedded pollen have no scattering anomalies, but part of the spectra is unusable, because of absorbance of the embedding matrix (paraffin). The signal processing algorithm removes most of the scatter components and allows the scatter components to be extracted. Classification of the different data-sets (spectra without correction, embedded spectra, processed spectra, scatter parameters) reveals that scatter correction methods reduce classification success and that scatter parameters contain taxonomic information. This suggests that scatter corrections may not be the best approach for applications mainly focused on classification or identification, while reconstructions of, for example, UV-B radiation may benefit from scatter correction methods, when measuring single grain spectra. This thesis shows that the performance of IR methods surpasses traditional morphological methods for pollen classification and that a considerable amount of taxonomic information is stored in functional groups associated with sporopollenin (phenylpropanoids). In a study on fossil pollen, this thesis demonstrates that conventional chemical extraction methods, such as acetolysis, alter the chemical composition of pollen and may not be ideal for palaeochemical purposes. Additionally, the scatter correction methods show that IR can provide non-chemical information in the form of scatter parameters, which contain taxonomic information. These results are useful additions to the growing knowledge on chemical methods for palaeoecological and palynological analyses.Doktorgradsavhandlin

Real-time pollen monitoring using digital holography

We present the first validation of the SwisensPoleno, currently the only operational automatic pollen mon-itoring system based on digital holography. The device pro-vides in-flight images of all coarse aerosols, and here wedevelop a two-step classification algorithm that uses theseimages to identify a range of pollen taxa. Deterministiccriteria based on the shape of the particle are applied toinitially distinguish between intact pollen grains and othercoarse particulate matter. This first level of discriminationidentifies pollen with an accuracy of 96 %. Thereafter, in-dividual pollen taxa are recognized using supervised learn-ing techniques. The algorithm is trained using data obtainedby inserting known pollen types into the device, and out ofeight pollen taxa six can be identified with an accuracy ofabove 90 %. In addition to the ability to correctly identifyaerosols, an automatic pollen monitoring system needs to beable to correctly determine particle concentrations. To fur-ther verify the device, controlled chamber experiments us-ing polystyrene latex beads were performed. This providedreference aerosols with traceable particle size and numberconcentrations in order to ensure particle size and samplingvolume were correctly characterized

Understandting soft matter through the use of novel biophysical tools

Pollen detection is an ongoing topic reviewed nowadays. This substance's impact on human healthcare is high, causing diseases, allergies and asthma. The principal limitation is the necessity of post-processing the laboratory. In this project, we propose to use a not usual tool in this field. With its high-information images, Holography may give us the difference when trying to recognize pollen grains. Combined with the powerful processing technique of computer vision, this project could be the first step in the in-live recognition of pollen. It can be ampliated to recognize another type of aerosols