Spatially Aware Dictionary Learning and Coding for Fossil Pollen Identification

We propose a robust approach for performing automatic species-level recognition of fossil pollen grains in microscopy images that exploits both global shape and local texture characteristics in a patch-based matching methodology. We introduce a novel criteria for selecting meaningful and discriminative exemplar patches. We optimize this function during training using a greedy submodular function optimization framework that gives a near-optimal solution with bounded approximation error. We use these selected exemplars as a dictionary basis and propose a spatially-aware sparse coding method to match testing images for identification while maintaining global shape correspondence. To accelerate the coding process for fast matching, we introduce a relaxed form that uses spatially-aware soft-thresholding during coding. Finally, we carry out an experimental study that demonstrates the effectiveness and efficiency of our exemplar selection and classification mechanisms, achieving $86.13\%$ accuracy on a difficult fine-grained species classification task distinguishing three types of fossil spruce pollen.Comment: CVMI 201

Pollen grain recognition through deep learning convolutional neural networks

Palynology is the study of pollen, in particular, the pollen’s grain type, but the tasks of classification and counting of pollen grains are highly skilled and laborious. Despite the efforts made during the last decades, the manual classification process is still predominant. One of the reasons for that is the small number of taxa usually used in previous approaches. In this paper, we propose a new method to automatically classify pollen grains using a state-of-the-art deep learning technique applied to the recently published POLEN73S image dataset. Our proposal manages to classify up to 94% of the samples from the dataset with 73 different classes of pollen grains. This result, which surpasses all previous attempts in number and difficulty of taxa under consideration, gives good perspectives to achieve a perfect score in pollen recognition task even with a large number of pollen grain types.info:eu-repo/semantics/publishedVersio

Automatic pollen recognition using convolutional neural networks: The case of the main pollens present in Spanish citrus and rosemary honey

[EN] The automation of honey pollen visual sorting overcomes the limitations of the conventional procedure helping the specialist in this time-consuming task. In this work, a novel and comprehensive Ground Truth of almost 19,000 images (from optical microscopy) of the 16 most abundant types of grains/pollen particles present in citrus and rosemary honey from Spain was constructed. This task was assisted by a HoneyApp (also developed herein) for the labelling and annotation process. Subsequently, the effectiveness of different pre-existing automatic pollen recognizers based on convolutional neural networks (CNN) (VGG16, VGG19, InceptionV3, Xception, ResNet50, DenseNet201, MobileNetV2 and EfficientNetV2M) was tested together with a new network proposed in this paper (PolleNetV1). The extreme complexity of those pre-existing CNN and extensive use of millions of parameters makes this new proposal especially promising. Although with a slightly lower accuracy (average 96%) in determining the relative frequencies of different types of pollen grains/particles, it has considerable advantages such as simplicity and ability to be included in the future functionality to automate pollen recognition in honey. This is the first step to finally achieving an objective tool that allows the correct labelling of any types of pollen in honey, thus contributing to its transparency in the market.This work is part of Spanish project PID2019-106800RB-I00 (2019) with financial support from the Ministerio de Ciencia e Innovacion (MCIN), Agencia Estatal de Investigacion MCIN/AEI/10.13039/501100011033/. It has been also part of the project AGROALNEXT/2022/043, funded by the Next Generation European Union and the Plan de Recuperacion, Transformacion y Resiliencia of the Spanish Government, with the support of Generalitat Valenciana. The authors would like to thank the CRUE-Universitat Politecnica deValencia for providing the funds for open access publication.Valiente González, JM.; Juan-Borras, MDS.; López García, F.; Escriche Roberto, MI. (2023). Automatic pollen recognition using convolutional neural networks: The case of the main pollens present in Spanish citrus and rosemary honey. Journal of Food Composition and Analysis. 123:1-10. https://doi.org/10.1016/j.jfca.2023.10560511012

Addressing subjectivity in the classification of palaeoenvironmental remains with supervised deep learning convolutional neural networks

Archaeological object identifications have been traditionally undertaken through a comparative methodology where each artefact is identified through a subjective, interpretative act by a professional. Regarding palaeoenvironmental remains, this comparative methodology is given boundaries by using reference materials and codified sets of rules, but subjectivity is nevertheless present. The problem with this traditional archaeological methodology is that higher level of subjectivity in the identification of artefacts leads to inaccuracies, which then increases the potential for Type I and Type II errors in the testing of hypotheses. Reducing the subjectivity of archaeological identifications would improve the statistical power of archaeological analyses, which would subsequently lead to more impactful research. In this thesis, it is shown that the level of subjectivity in palaeoenvironmental research can be reduced by applying deep learning convolutional neural networks within an image recognition framework. The primary aim of the presented research is therefore to further the on-going paradigm shift in archaeology towards model-based object identifications, particularly within the realm of palaeoenvironmental remains. Although this thesis focuses on the identification of pollen grains and animal bones, with the latter being restricted to the astragalus of sheep and goats, there are wider implications for archaeology as these methods can easily be extended beyond pollen and animal remains. The previously published POLEN23E dataset is used as the pilot study of applying deep learning in pollen grain classification. In contrast, an image dataset of modern bones was compiled for the classification of sheep and goat astragali due to a complete lack of available bone image datasets and a double blind study with inexperienced and experienced zooarchaeologists was performed to have a benchmark to which image recognition models can be compared. In both classification tasks, the presented models outperform all previous formal modelling methods and only the best human analysts match the performance of the deep learning model in the sheep and goat astragalus separation task. Throughout the thesis, there is a specific focus on increasing trust in the models through the visualization of the models’ decision making and avenues of improvements to Grad-CAM are explored. This thesis makes an explicit case for the phasing out of the comparative methods in favour of a formal modelling framework within archaeology, especially in palaeoenvironmental object identification

Understandting soft matter through the use of novel biophysical tools

Pollen detection is an ongoing topic reviewed nowadays. This substance's impact on human healthcare is high, causing diseases, allergies and asthma. The principal limitation is the necessity of post-processing the laboratory. In this project, we propose to use a not usual tool in this field. With its high-information images, Holography may give us the difference when trying to recognize pollen grains. Combined with the powerful processing technique of computer vision, this project could be the first step in the in-live recognition of pollen. It can be ampliated to recognize another type of aerosols

РАСПОЗНАВАНИЕ МИКРОСКОПИЧЕСКИХ ИЗОБРАЖЕНИЙ ПЫЛЬЦЕВЫХ ЗЕРЕН С ПОМОЩЬЮ СВЕРТОЧНОЙ НЕЙРОННОЙ СЕТИ VGG-16

The article presents the result of an experiment on the application of transfer learning using the Visual Geometry Group with 16 layers (VGG-16) convolutional neural network in relation to the problem of recognizing pollen grains in images. An analysis of the information-theoretical base on the application of machine learning algorithms to the problem of classifying pollen grains over the past few years has shown the need to develop (apply) a new method for recognizing images of pollen grains obtained using an optical microscope. Currently, automatic classification for pollen identification is becoming a very active area of research. The article substantiates the task of automating the classification of pollen grains. The aim of the study is to analyze the efficiency and accuracy of classifying microscopic images of pollen grains using transfer learning of the VGG-16 convolutional neural network. Transfer learning was performed using the VGG-16 neural network, which has 13 convolutional layers grouped into 5 blocks with pooling and 3 smoothing layers at the output. Since transfer learning is used, the number of training epochs can be chosen to be small. For this network, only the smoothing output layers change, and the feature extraction remains the same as in the classical model. Therefore, it was chosen to use 10 training epochs. Other hyperparameters are as follows: Drop Out regularization with a probability of 50%, optimization method is ADAM, activation function is sigmoid, loss function is cross-entropy, batch size is 32 images. As a result, by adjusting the hyperparameters of the model and using augmentation, it was possible to achieve a share of correct recognitions of about 80%. At the same time, due to the different number of training examples, the particular characteristics of the classes differ somewhat. Thus, the maximum precision and recall reach 94% and 83%, respectively, for the Dandelion type. In the future, studies are planned to use augmentation as a preprocessing to create a balanced sample. By applying the VGG-16 convolutional neural network to the problem of pollen grain image recognition, high accuracy and efficiency of the method were achieved.В статье приводится результат эксперимента по применению трансферного обучения с помощью сверточной нейронной сети Visual Geometry Group with 16 layers (VGG-16) применительно к задаче распознавания пыльцевых зерен на изображениях. Анализ информационно-теоретической базы по применению алгоритмов машинного обучения к задаче классификации пыльцевых зерен за последние несколько лет показал необходимость разработки (применения) нового метода к распознаванию изображений пыльцевых зерен, полученных с помощью оптического микроскопа. В настоящее время автоматическая классификация для идентификации пыльцы становится очень активной областью исследований. В статье обоснована задача автоматизации классификации пыльцевых зерен. Целью исследования является анализ эффективности и точности классификации микроскопических изображений пыльцевых зерен с помощью трансферного обучения сверточной нейронной сети VGG-16. Трансферное обучение было выполнено с помощью нейронной сети VGG-16, имеющей 13 сверточных слоев, группируемых в 5 блоков с пулингом и 3 сглаживающих слоя на выходе. Поскольку применяется трансферное обучение, то количество эпох обучения можно выбрать небольшим. У данной сети меняются только сглаживающие выходные слои, а извлечение признаков осуществляется с весами классической модели. Поэтому было выбрано использовать 10 эпох обучения. Другие гиперпараметры – регуляризация Drop Out с вероятностью 50 %, метод оптимизации – ADAM, функция активации – sigmoid, функция потерь – кросс-энтропия, размер батча – 32 изображения. В результате за счет настройки гиперпараметров модели и использования аугментаций удалось достичь доли верных распознаваний порядка 80 %. При этом в связи с разным количеством обучающих примеров частные характеристики по классам несколько отличаются. Так, максимальные точность и полнота достигают 94 и 83 % соответственно для типа Одуванчик. В будущем планируются исследования для применения аугментации в качестве предобработки для создания сбалансированной выборки. За счет применения сверточной нейронной сети VGG-16 к задаче распознаваний изображений пыльцевых зерен были достигнуты высокие показатели точности и эффективности метода

New methods in Palaeopalynology: Classification of pollen through pollen chemistry

Pollen grains are one of the primary tools of palaeoecologists to reconstruct vegetation changes in the past. The description, counting and analysis of pollen grains (palynology) has contributed to our understanding of establishment and dynamics of past and present plant communities. Advances in identification accuracy, precision and increased taxonomic resolution have greatly improved our understanding of biogeography and plant community interactions. Nevertheless, the techniques by which palynological studies are performed have not fundamentally changed. Taxonomic resolution and automation have been identified as some of the key challenges for palynology and palaeoecology. Chemical methods have been proposed as a potential alternative to morphological approaches and have demonstrated promising results in the classification of modern pollen grains and in the analysis of pollen chemical responses to UV-B radiation. The application of chemical methods for palynological needs have not been thoroughly explored, with analysis of (sub-)fossil pollen lagging behind their modern counterpart. Especially the application of infrared methods have gained popularity as an alternative to traditional morphological approaches. In this thesis, I explore the use of infrared methods for palynological applications, by exploring the chemical variation in modern pollen grains and in the analysis of fossil pollen grains with IR microscope approaches. The objectives of this thesis are formulated into three research objectives: * Collect modern pollen and explore the variation in chemical composition * Apply chemical methods to fossil material * Explore microscopy chemical methods on modern pollen The thesis is structured into four studies to study these objectives. Papers I and II explore variation and classification based on the chemical composition of modern *Quercus* pollen using two IR approaches, Fourier transform infrared spectroscopy (FTIR) and Fourier transform Raman spectroscopy (FT-Raman). After exploring modern chemical composition of pollen, paper III investigates FTIR methods for the analysis of fossil pollen, in spectra of Holocene *Pinus* pollen. Additionally, the effects of acetolysis and density separation on *Pinus* pollen is described. Paper IV addresses the challenge of scattering signals when measuring small pollen grains of four *Quercus* species with FTIR microscopy and ways to surpress or weaken the scattering signals. The results from paper I and II show classification success, surpassing traditional morphological approaches, at the *Quercus* section level and ~90% recall on species level with both IR approaches. Chemical bands most useful for classification are lipids, sporopollenin and proteins for both FT-Raman and FTIR. We observe differences in the importance of chemical functional groups for the classification. FT-Raman relies more on sporopollenin chemistry, while FTIR utilizes more variation in lipid bands. After finding considerable variation in sporopollenin chemistry in modern pollen samples, FTIR methods were applied to pollen from sediment cores spanning the Holocene. Paper III examines the differences between modern and sub-fossil pollen and reported large differences between them, mainly the removal of labile components, such as lipids and protein peaks from the sub-fossil spectra during diagenesis. Additionally, paper III finds changes to pollen chemistry caused by acetolysis in the 1200 - 1000 cm^-1^ region of the spectra, when comparing acetolysed spectra to non-acetolysed spectra. The paper concludes with findings of unwanted inorganic signals (BSi) and contamination from density separation media in the sediment pollen spectra. Paper IV demonstrates two successful methods of removing scattering signals from pollen spectra. Two approaches were examined, embedding and processing with signal correction algorithms. Spectra from embedded pollen have no scattering anomalies, but part of the spectra is unusable, because of absorbance of the embedding matrix (paraffin). The signal processing algorithm removes most of the scatter components and allows the scatter components to be extracted. Classification of the different data-sets (spectra without correction, embedded spectra, processed spectra, scatter parameters) reveals that scatter correction methods reduce classification success and that scatter parameters contain taxonomic information. This suggests that scatter corrections may not be the best approach for applications mainly focused on classification or identification, while reconstructions of, for example, UV-B radiation may benefit from scatter correction methods, when measuring single grain spectra. This thesis shows that the performance of IR methods surpasses traditional morphological methods for pollen classification and that a considerable amount of taxonomic information is stored in functional groups associated with sporopollenin (phenylpropanoids). In a study on fossil pollen, this thesis demonstrates that conventional chemical extraction methods, such as acetolysis, alter the chemical composition of pollen and may not be ideal for palaeochemical purposes. Additionally, the scatter correction methods show that IR can provide non-chemical information in the form of scatter parameters, which contain taxonomic information. These results are useful additions to the growing knowledge on chemical methods for palaeoecological and palynological analyses.Doktorgradsavhandlin

Neural networks for increased accuracy of allergenic pollen monitoring

Computer Systems, Imagery and Medi