EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020

Welcome to EVALITA 2020! EVALITA is the evaluation campaign of Natural Language Processing and Speech Tools for Italian. EVALITA is an initiative of the Italian Association for Computational Linguistics (AILC, http://www.ai-lc.it) and it is endorsed by the Italian Association for Artificial Intelligence (AIxIA, http://www.aixia.it) and the Italian Association for Speech Sciences (AISV, http://www.aisv.it)

The text classification pipeline: Starting shallow, going deeper

An increasingly relevant and crucial subfield of Natural Language Processing (NLP), tackled in this PhD thesis from a computer science and engineering perspective, is the Text Classification (TC). Also in this field, the exceptional success of deep learning has sparked a boom over the past ten years. Text retrieval and categorization, information extraction and summarization all rely heavily on TC. The literature has presented numerous datasets, models, and evaluation criteria. Even if languages as Arabic, Chinese, Hindi and others are employed in several works, from a computer science perspective the most used and referred language in the literature concerning TC is English. This is also the language mainly referenced in the rest of this PhD thesis. Even if numerous machine learning techniques have shown outstanding results, the classifier effectiveness depends on the capability to comprehend intricate relations and non-linear correlations in texts. In order to achieve this level of understanding, it is necessary to pay attention not only to the architecture of a model but also to other stages of the TC pipeline. In an NLP framework, a range of text representation techniques and model designs have emerged, including the large language models. These models are capable of turning massive amounts of text into useful vector representations that effectively capture semantically significant information. The fact that this field has been investigated by numerous communities, including data mining, linguistics, and information retrieval, is an aspect of crucial interest. These communities frequently have some overlap, but are mostly separate and do their research on their own. Bringing researchers from other groups together to improve the multidisciplinary comprehension of this field is one of the objectives of this dissertation. Additionally, this dissertation makes an effort to examine text mining from both a traditional and modern perspective. This thesis covers the whole TC pipeline in detail. However, the main contribution is to investigate the impact of every element in the TC pipeline to evaluate the impact on the final performance of a TC model. It is discussed the TC pipeline, including the traditional and the most recent deep learning-based models. This pipeline consists of State-Of-The-Art (SOTA) datasets used in the literature as benchmark, text preprocessing, text representation, machine learning models for TC, evaluation metrics and current SOTA results. In each chapter of this dissertation, I go over each of these steps, covering both the technical advancements and my most significant and recent findings while performing experiments and introducing novel models. The advantages and disadvantages of various options are also listed, along with a thorough comparison of the various approaches. At the end of each chapter, there are my contributions with experimental evaluations and discussions on the results that I have obtained during my three years PhD course. The experiments and the analysis related to each chapter (i.e., each element of the TC pipeline) are the main contributions that I provide, extending the basic knowledge of a regular survey on the matter of TC.An increasingly relevant and crucial subfield of Natural Language Processing (NLP), tackled in this PhD thesis from a computer science and engineering perspective, is the Text Classification (TC). Also in this field, the exceptional success of deep learning has sparked a boom over the past ten years. Text retrieval and categorization, information extraction and summarization all rely heavily on TC. The literature has presented numerous datasets, models, and evaluation criteria. Even if languages as Arabic, Chinese, Hindi and others are employed in several works, from a computer science perspective the most used and referred language in the literature concerning TC is English. This is also the language mainly referenced in the rest of this PhD thesis. Even if numerous machine learning techniques have shown outstanding results, the classifier effectiveness depends on the capability to comprehend intricate relations and non-linear correlations in texts. In order to achieve this level of understanding, it is necessary to pay attention not only to the architecture of a model but also to other stages of the TC pipeline. In an NLP framework, a range of text representation techniques and model designs have emerged, including the large language models. These models are capable of turning massive amounts of text into useful vector representations that effectively capture semantically significant information. The fact that this field has been investigated by numerous communities, including data mining, linguistics, and information retrieval, is an aspect of crucial interest. These communities frequently have some overlap, but are mostly separate and do their research on their own. Bringing researchers from other groups together to improve the multidisciplinary comprehension of this field is one of the objectives of this dissertation. Additionally, this dissertation makes an effort to examine text mining from both a traditional and modern perspective. This thesis covers the whole TC pipeline in detail. However, the main contribution is to investigate the impact of every element in the TC pipeline to evaluate the impact on the final performance of a TC model. It is discussed the TC pipeline, including the traditional and the most recent deep learning-based models. This pipeline consists of State-Of-The-Art (SOTA) datasets used in the literature as benchmark, text preprocessing, text representation, machine learning models for TC, evaluation metrics and current SOTA results. In each chapter of this dissertation, I go over each of these steps, covering both the technical advancements and my most significant and recent findings while performing experiments and introducing novel models. The advantages and disadvantages of various options are also listed, along with a thorough comparison of the various approaches. At the end of each chapter, there are my contributions with experimental evaluations and discussions on the results that I have obtained during my three years PhD course. The experiments and the analysis related to each chapter (i.e., each element of the TC pipeline) are the main contributions that I provide, extending the basic knowledge of a regular survey on the matter of TC

Addressing subjectivity in the classification of palaeoenvironmental remains with supervised deep learning convolutional neural networks

Archaeological object identifications have been traditionally undertaken through a comparative methodology where each artefact is identified through a subjective, interpretative act by a professional. Regarding palaeoenvironmental remains, this comparative methodology is given boundaries by using reference materials and codified sets of rules, but subjectivity is nevertheless present. The problem with this traditional archaeological methodology is that higher level of subjectivity in the identification of artefacts leads to inaccuracies, which then increases the potential for Type I and Type II errors in the testing of hypotheses. Reducing the subjectivity of archaeological identifications would improve the statistical power of archaeological analyses, which would subsequently lead to more impactful research. In this thesis, it is shown that the level of subjectivity in palaeoenvironmental research can be reduced by applying deep learning convolutional neural networks within an image recognition framework. The primary aim of the presented research is therefore to further the on-going paradigm shift in archaeology towards model-based object identifications, particularly within the realm of palaeoenvironmental remains. Although this thesis focuses on the identification of pollen grains and animal bones, with the latter being restricted to the astragalus of sheep and goats, there are wider implications for archaeology as these methods can easily be extended beyond pollen and animal remains. The previously published POLEN23E dataset is used as the pilot study of applying deep learning in pollen grain classification. In contrast, an image dataset of modern bones was compiled for the classification of sheep and goat astragali due to a complete lack of available bone image datasets and a double blind study with inexperienced and experienced zooarchaeologists was performed to have a benchmark to which image recognition models can be compared. In both classification tasks, the presented models outperform all previous formal modelling methods and only the best human analysts match the performance of the deep learning model in the sheep and goat astragalus separation task. Throughout the thesis, there is a specific focus on increasing trust in the models through the visualization of the models’ decision making and avenues of improvements to Grad-CAM are explored. This thesis makes an explicit case for the phasing out of the comparative methods in favour of a formal modelling framework within archaeology, especially in palaeoenvironmental object identification

Remote Sensing of Plant Biodiversity

At last, here it is. For some time now, the world has needed a text providing both a new theoretical foundation and practical guidance on how to approach the challenge of biodiversity decline in the Anthropocene. This is a global challenge demanding global approaches to understand its scope and implications. Until recently, we have simply lacked the tools to do so. We are now entering an era in which we can realistically begin to understand and monitor the multidimensional phenomenon of biodiversity at a planetary scale. This era builds upon three centuries of scientific research on biodiversity at site to landscape levels, augmented over the past two decades by airborne research platforms carrying spectrometers, lidars, and radars for larger-scale observations. Emerging international networks of fine-grain in-situ biodiversity observations complemented by space-based sensors offering coarser-grain imagery—but global coverage—of ecosystem composition, function, and structure together provide the information necessary to monitor and track change in biodiversity globally. This book is a road map on how to observe and interpret terrestrial biodiversity across scales through plants—primary producers and the foundation of the trophic pyramid. It honors the fact that biodiversity exists across different dimensions, including both phylogenetic and functional. Then, it relates these aspects of biodiversity to another dimension, the spectral diversity captured by remote sensing instruments operating at scales from leaf to canopy to biome. The biodiversity community has needed a Rosetta Stone to translate between the language of satellite remote sensing and its resulting spectral diversity and the languages of those exploring the phylogenetic diversity and functional trait diversity of life on Earth. By assembling the vital translation, this volume has globalized our ability to track biodiversity state and change. Thus, a global problem meets a key component of the global solution. The editors have cleverly built the book in three parts. Part 1 addresses the theory behind the remote sensing of terrestrial plant biodiversity: why spectral diversity relates to plant functional traits and phylogenetic diversity. Starting with first principles, it connects plant biochemistry, physiology, and macroecology to remotely sensed spectra and explores the processes behind the patterns we observe. Examples from the field demonstrate the rising synthesis of multiple disciplines to create a new cross-spatial and spectral science of biodiversity. Part 2 discusses how to implement this evolving science. It focuses on the plethora of novel in-situ, airborne, and spaceborne Earth observation tools currently and soon to be available while also incorporating the ways of actually making biodiversity measurements with these tools. It includes instructions for organizing and conducting a field campaign. Throughout, there is a focus on the burgeoning field of imaging spectroscopy, which is revolutionizing our ability to characterize life remotely. Part 3 takes on an overarching issue for any effort to globalize biodiversity observations, the issue of scale. It addresses scale from two perspectives. The first is that of combining observations across varying spatial, temporal, and spectral resolutions for better understanding—that is, what scales and how. This is an area of ongoing research driven by a confluence of innovations in observation systems and rising computational capacity. The second is the organizational side of the scaling challenge. It explores existing frameworks for integrating multi-scale observations within global networks. The focus here is on what practical steps can be taken to organize multi-scale data and what is already happening in this regard. These frameworks include essential biodiversity variables and the Group on Earth Observations Biodiversity Observation Network (GEO BON). This book constitutes an end-to-end guide uniting the latest in research and techniques to cover the theory and practice of the remote sensing of plant biodiversity. In putting it together, the editors and their coauthors, all preeminent in their fields, have done a great service for those seeking to understand and conserve life on Earth—just when we need it most. For if the world is ever to construct a coordinated response to the planetwide crisis of biodiversity loss, it must first assemble adequate—and global—measures of what we are losing