A Comprehensive Survey of Deep Learning in Remote Sensing: Theories, Tools and Challenges for the Community

In recent years, deep learning (DL), a re-branding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, natural language processing, etc. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV; e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should be aware of, if not at the leading edge of, of advancements like DL. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as it relates to (i) inadequate data sets, (ii) human-understandable solutions for modelling physical phenomena, (iii) Big Data, (iv) non-traditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.Comment: 64 pages, 411 references. To appear in Journal of Applied Remote Sensin

An under-ice hyperspectral and RGB imaging system to capture fine-scale biophysical properties of sea ice

Sea-ice biophysical properties are characterized by high spatio-temporal variability ranging from the meso- to the millimeter scale. Ice coring is a common yet coarse point sampling technique that struggles to capture such variability in a non-invasive manner. This hinders quantification and understanding of ice algae biomass patchiness and its complex interaction with some of its sea ice physical drivers. In response to these limitations, a novel under-ice sled system was designed to capture proxies of biomass together with 3D models of bottom topography of land-fast sea-ice. This system couples a pushbroom hyperspectral imaging (HI) sensor with a standard digital RGB camera and was trialed at Cape Evans, Antarctica. HI aims to quantify per-pixel chlorophyll-a content and other ice algae biological properties at the ice-water interface based on light transmitted through the ice. RGB imagery processed with digital photogrammetry aims to capture under-ice structure and topography. Results from a 20 m transect capturing a 0.61 m wide swath at sub-mm spatial resolution are presented. We outline the technical and logistical approach taken and provide recommendations for future deployments and developments of similar systems. A preliminary transect subsample was processed using both established and novel under-ice bio-optical indices (e.g., normalized difference indexes and the area normalized by the maximal band depth) and explorative analyses (e.g., principal component analyses) to establish proxies of algal biomass. This first deployment of HI and digital photogrammetry under-ice provides a proof-of-concept of a novel methodology capable of delivering non-invasive and highly resolved estimates of ice algal biomass in-situ, together with some of its environmental drivers. Nonetheless, various challenges and limitations remain before our method can be adopted across a range of sea-ice conditions. Our work concludes with suggested solutions to these challenges and proposes further method and system developments for future research

The EnMAP Managed Vegetation Scientific Processor

Nach jahrelanger wissenschaftlicher und technischer Vorbereitungszeit wird voraussichtlich Ende des Jahres 2020 der Start der orbitalen Phase einer unbemannten deutschen Weltraum-Mission initiiert. Das Environmental Mapping and Analysis Program (EnMAP) wird an Bord des gleichnamigen Satelliten einen hyperspektralen Sensor zur Erfassung terrestrischer Oberflächen tragen. In den Umweltdisziplinen zur Erforschung von Ökosystemen, landwirtschaftlicher, forstwirtschaftlicher und urbaner Flächen, im Bereich der Küsten- und Inlandsgewässer sowie der Geologie und Bodenkunde bereitete man sich im Vorfeld des Starts auf die kommenden Daten vor. Zwar existiert bereits eine Vielzahl an Algorithmen zur wissenschaftlichen Analyse von spektralen Daten, allerdings ergeben sich auch neue Herausforderungen, da die EnMAP-Mission bislang im weltweiten Kontext der Fernerkundung einzigartig ist. Die Abdeckung des vollen optischen Spektrums (420 nm – 2450 nm) in Verbindung mit einer moderaten räumlichen Auflösung von 30 m und einem hohen Signal-Rausch-Verhältnis von mindestens 180 im kurzwelligen Infrarot und über 400 im sichtbaren Spektrum, ermöglichen eine Aufnahmequalität, die bislang nur von flugzeuggestützten Systemen erreicht werden konnte. Die Bemühungen in dieser Dissertation umfassen Aktivitäten in der wissenschaftlichen Vorbereitungsphase zu agrargeographischen Fragestellungen. Algorithmen und Tools zur Analyse der hyperspektralen Daten werden kostenlos im QGIS-Plugin EnMAP-Box 3 zur Verfügung gestellt. Die drängenden Fragen im Agrarsektor drehen sich hierbei um die Ableitung biochemischer und biophysikalischer Parameter aus Fernerkundungsdaten, weshalb die übergeordnete Problemstellung des Promotionsvorhabens die Entwicklung eines wissenschaftsbasierten EnMAP-Tools für bewirtschaftete Vegetationsflächen (EnMAP Managed Vegetation Scientific Processor) darstellt. Zu Beginn wurde eine umfassende Feldkampagne geplant, welche ab April 2014 umgesetzt wurde. Neben der spektralen Erfassung von Blatt-, Bestands- und Bodensignaturen in einem Winterweizen- und einem Maisfeld erfolgte auch die Messung wesentlicher Pflanzenparameter an den exakt gleichen Positionen. Hierzu zählt die non-destruktive Ableitung des Blattflächenindex (LAI), des Blattchlorophyllgehalts (Ccab), des Blattwassergehalts (EWT oder Cw), des relativen Blatttrockengewichts (LMA oder Cm), des mittleren Blattneigungswinkels im Bestand (ALIA) sowie weiterer sekundärer Parameter wie Wuchshöhe, das phänologisches Stadium und der Sonnenvektor. Um die Fähigkeit des späteren EnMAP-Satelliten sich um bis zu 30° orthogonal zur Flugrichtung zu kippen nachzustellen, wurden die spektralen Aufnahmen aus verschiedenen Betrachtungswinkeln erstellt, die dieser Aufnahme-Geometrien nachempfunden sind. Ein gängiges Verfahren zur Ableitung der relevanten Pflanzenparameter ist die Verwendung des Strahlungstransfermodells PROSAIL, welches das spektrale Signal einer Vegetationsfläche auf Basis der zugrundeliegenden biophysikalischen und biochemischen Parameter simuliert. Bei der Umkehr dieses Prozesses können ebendiese Variablen von gemessenen spektralen Daten abgeleitet werden. Hierzu wurde eine Datenbank (Look-Up-Table, LUT) aus PROSAIL-Modellläufen aufgebaut und die in den Feldkampagnen gemessenen Spektren mit dieser abgeglichen. Mit dieser Methode der LUT-Invertierung aus unterschiedlichen Aufnahmewinkeln konnten Genauigkeiten bei der LAI-Schätzung von 18 % und bei Blattchlorophyll von 20 % erzielt werden. Eine starke Anisotropie, also eine Reflexionsabhängigkeit von der Beleuchtungs- und Aufnahmerichtung, wurde bei Winterweizen vor allem für frühe Entwicklungsstadien festgestellt. Bei einer anschließenden Studie zur Unsicherheitsanalyse des Spektralmodells wurden PROSAIL-Ergebnisse, bei denen real gemessene Pflanzenparameter als Input dienten, den zugehörigen Reflektanzspektren gegenübergestellt. Es zeigten sich hierbei mitunter starke Abweichungen zwischen gemessenen und modellierten Spektren, die im Falle des Winterweizens einen saisonalen Verlauf zeichneten. Vor allem während frühen Wachstumsstadien tendierte das Modell dazu die Reflektanz im nahen Infrarot zu überschätzen, während es gegen Ende der Wachstumsperiode eher eine Unterschätzung aufwies. Als Unsicherheitsfaktor wurde die Parametrisierung des Modells ausgemacht, wenn der ALIA-Parameter als echter physikalische Blattwinkel interpretiert wird. Es wurde geschlussfolgert, dass eine Separierung von LAI und ALIA bei der Invertierung von PROSAIL eine korrekte Abschätzung der weniger sensitiven Parameter behindert. Die Erstellung des Vegetations-Prozessors erforderte die Verwendung von Regressions-Algorithmen des maschinellen Lernens (MLRA), da eine Verteilung von großen LUTs an die User nicht praktikabel wäre. Die MLRAs wurden an synthetischen Datensätzen trainiert, wobei zunächst die Optimierung der Hyperparameter im Vordergrund stand, bevor die Anwendung an echten Spektraldaten unternommen wurde. Es konnten dabei erst aussagekräftige Ergebnisse produziert werden, als die Trainingsdaten mit einem künstlichen Rauschen belegt wurden, da die Algorithmen unter einer Überanpassung an die Modellumgebung litten. Mithilfe des Prozessors konnten schließlich LAI, ALIA, Ccab und Cw aus hyperspektralen Daten abgeleitet werden. Künstliche neuronale Netze dienen dabei als Blackbox-Modelle, die in kurzer Zeit große Datenmengen verarbeiten können und somit einen entscheidenden Beitrag zur modernen angewandten Fernerkundung für eine breite User-Community leisten.After years of scientific and technical preparation, the launch of an unmanned German space-mission is planned to be initiated in 2020. The Environmental Mapping and Analysis Program (EnMAP) is going to provide an equally named hyperspectral imager to map land surfaces. Scientists of environmental disciplines of monitoring of ecosystems, agricultural, forestry and urban areas as well as coastal and inland waters, geology and soils prepared themselves for the upcoming data prior to the actual launch. Although there already exists a variety of useful algorithms for a profound analysis of spectral data, new challenges will arise given the uniqueness of the EnMAP-mission in the global context of remote sensing; i.e. coverage of the full range of the optical spectrum (420 nm – 2450 nm) in combination with a moderate spatial resolution of 30 m and a high signal-to-noise ratio of at least 180 in the shortwave infrared and above 400 in the visible spectrum. This enables an imaging quality which to this date has only been reached by airborne systems. The efforts of this dissertation comprise activities in the scientific preparation phase for agro-geographical tasks. Algorithms and tools for an analysis of hyperspectral data are being provided for free in the QGIS-plugin EnMAP-Box 3. Urgent questions in the agricultural sector revolve around the derivation of biochemical and biophysical parameters from remote sensing data. For this reason, the overarching objective of this promotion is the development of a scientific EnMAP-tool for managed areas of vegetation (EnMAP Managed Vegetation Scientific Processor). At first, an extensive field campaign was planned and then started in April, 2014. Apart from spectral observations of leaves, canopies and soils in a winter wheat and a maize field, also relevant plant parameters were acquired at the exact same spots. Namely, they are the Leaf Area Index (LAI), leaf chlorophyll content (Ccab), leaf water content (EWT or Cw), relative dry leaf weight (LMA or Cm), Average Leaf Inclination Angle (ALIA) as well as other secondary parameters like canopy height, phenological stage and the solar vector. Spectral measurements were captured from different observation angles to match ground data with the sensing geometry of the future EnMAP-satellite, which can be tilted up to 30° orthogonal to its direction of flight. A common procedure to derive relevant crop parameters is to make use of the radiative transfer model PROSAIL, which simulates the spectral signal of a vegetated surface based on biophysical and biochemical input parameters. If this process is reverted, said parameters can be derived from measured spectral data. To do so, a Look-Up-Table (LUT) is built containing model runs of PROSAIL and then subsequently compared against spectra from the field campaigns. With this approach of LUT-inversions from different observation angles, an accuracy of 18 % could be achieved for LAI and 20 % for Ccab. Strong anisotropic effects, i.e. dependence on illumination geometry and sensor orientation, were identified for winter wheat mainly in the early stages of plant development. In a consecutive study about uncertainties of the spectral model, PROSAIL results fed with in situ measured crop parameters as input, were opposed to their associated reflectance signatures. A strong deviation between measured and modelled spectra was observed, which – in the case of winter wheat – showed a seasonal behavior. The model tended to overestimate reflectances in the near infrared for early phenological stages and to underestimate them at end of the growing period. The parametrization of the model was identified as an uncertainty factor if the ALIA parameter is interpreted as true physical leaf inclinations. It was concluded that a separation of LAI and ALIA at inversion of PROSAIL prevents an adequate estimation of the less sensitive parameters. The development of the vegetation processor required the use of Machine Learning Regression Algorithms (MLRA), since distribution of large LUTs to the user would be impracticable. The MLRAs were trained with synthetic datasets with primary importance to optimize their hyperparameters, before attempting to apply the algorithms to real spectral data. Significant results could not be obtained until training data were altered with artificial noise, because algorithms suffered from overfitting to the model environment. Executing the processor allowed to derive LAI, ALIA, Ccab and Cw from hyperspectral data. Artificial neural networks served as black box models, which digest great amount of data in a short period of time and thus make a decisive contribution to modern applied remote sensing with relevance for a broad user-community

The EnMAP Managed Vegetation Scientific Processor

Nach jahrelanger wissenschaftlicher und technischer Vorbereitungszeit wird voraussichtlich Ende des Jahres 2020 der Start der orbitalen Phase einer unbemannten deutschen Weltraum-Mission initiiert. Das Environmental Mapping and Analysis Program (EnMAP) wird an Bord des gleichnamigen Satelliten einen hyperspektralen Sensor zur Erfassung terrestrischer Oberflächen tragen. In den Umweltdisziplinen zur Erforschung von Ökosystemen, landwirtschaftlicher, forstwirtschaftlicher und urbaner Flächen, im Bereich der Küsten- und Inlandsgewässer sowie der Geologie und Bodenkunde bereitete man sich im Vorfeld des Starts auf die kommenden Daten vor. Zwar existiert bereits eine Vielzahl an Algorithmen zur wissenschaftlichen Analyse von spektralen Daten, allerdings ergeben sich auch neue Herausforderungen, da die EnMAP-Mission bislang im weltweiten Kontext der Fernerkundung einzigartig ist. Die Abdeckung des vollen optischen Spektrums (420 nm – 2450 nm) in Verbindung mit einer moderaten räumlichen Auflösung von 30 m und einem hohen Signal-Rausch-Verhältnis von mindestens 180 im kurzwelligen Infrarot und über 400 im sichtbaren Spektrum, ermöglichen eine Aufnahmequalität, die bislang nur von flugzeuggestützten Systemen erreicht werden konnte. Die Bemühungen in dieser Dissertation umfassen Aktivitäten in der wissenschaftlichen Vorbereitungsphase zu agrargeographischen Fragestellungen. Algorithmen und Tools zur Analyse der hyperspektralen Daten werden kostenlos im QGIS-Plugin EnMAP-Box 3 zur Verfügung gestellt. Die drängenden Fragen im Agrarsektor drehen sich hierbei um die Ableitung biochemischer und biophysikalischer Parameter aus Fernerkundungsdaten, weshalb die übergeordnete Problemstellung des Promotionsvorhabens die Entwicklung eines wissenschaftsbasierten EnMAP-Tools für bewirtschaftete Vegetationsflächen (EnMAP Managed Vegetation Scientific Processor) darstellt. Zu Beginn wurde eine umfassende Feldkampagne geplant, welche ab April 2014 umgesetzt wurde. Neben der spektralen Erfassung von Blatt-, Bestands- und Bodensignaturen in einem Winterweizen- und einem Maisfeld erfolgte auch die Messung wesentlicher Pflanzenparameter an den exakt gleichen Positionen. Hierzu zählt die non-destruktive Ableitung des Blattflächenindex (LAI), des Blattchlorophyllgehalts (Ccab), des Blattwassergehalts (EWT oder Cw), des relativen Blatttrockengewichts (LMA oder Cm), des mittleren Blattneigungswinkels im Bestand (ALIA) sowie weiterer sekundärer Parameter wie Wuchshöhe, das phänologisches Stadium und der Sonnenvektor. Um die Fähigkeit des späteren EnMAP-Satelliten sich um bis zu 30° orthogonal zur Flugrichtung zu kippen nachzustellen, wurden die spektralen Aufnahmen aus verschiedenen Betrachtungswinkeln erstellt, die dieser Aufnahme-Geometrien nachempfunden sind. Ein gängiges Verfahren zur Ableitung der relevanten Pflanzenparameter ist die Verwendung des Strahlungstransfermodells PROSAIL, welches das spektrale Signal einer Vegetationsfläche auf Basis der zugrundeliegenden biophysikalischen und biochemischen Parameter simuliert. Bei der Umkehr dieses Prozesses können ebendiese Variablen von gemessenen spektralen Daten abgeleitet werden. Hierzu wurde eine Datenbank (Look-Up-Table, LUT) aus PROSAIL-Modellläufen aufgebaut und die in den Feldkampagnen gemessenen Spektren mit dieser abgeglichen. Mit dieser Methode der LUT-Invertierung aus unterschiedlichen Aufnahmewinkeln konnten Genauigkeiten bei der LAI-Schätzung von 18 % und bei Blattchlorophyll von 20 % erzielt werden. Eine starke Anisotropie, also eine Reflexionsabhängigkeit von der Beleuchtungs- und Aufnahmerichtung, wurde bei Winterweizen vor allem für frühe Entwicklungsstadien festgestellt. Bei einer anschließenden Studie zur Unsicherheitsanalyse des Spektralmodells wurden PROSAIL-Ergebnisse, bei denen real gemessene Pflanzenparameter als Input dienten, den zugehörigen Reflektanzspektren gegenübergestellt. Es zeigten sich hierbei mitunter starke Abweichungen zwischen gemessenen und modellierten Spektren, die im Falle des Winterweizens einen saisonalen Verlauf zeichneten. Vor allem während frühen Wachstumsstadien tendierte das Modell dazu die Reflektanz im nahen Infrarot zu überschätzen, während es gegen Ende der Wachstumsperiode eher eine Unterschätzung aufwies. Als Unsicherheitsfaktor wurde die Parametrisierung des Modells ausgemacht, wenn der ALIA-Parameter als echter physikalische Blattwinkel interpretiert wird. Es wurde geschlussfolgert, dass eine Separierung von LAI und ALIA bei der Invertierung von PROSAIL eine korrekte Abschätzung der weniger sensitiven Parameter behindert. Die Erstellung des Vegetations-Prozessors erforderte die Verwendung von Regressions-Algorithmen des maschinellen Lernens (MLRA), da eine Verteilung von großen LUTs an die User nicht praktikabel wäre. Die MLRAs wurden an synthetischen Datensätzen trainiert, wobei zunächst die Optimierung der Hyperparameter im Vordergrund stand, bevor die Anwendung an echten Spektraldaten unternommen wurde. Es konnten dabei erst aussagekräftige Ergebnisse produziert werden, als die Trainingsdaten mit einem künstlichen Rauschen belegt wurden, da die Algorithmen unter einer Überanpassung an die Modellumgebung litten. Mithilfe des Prozessors konnten schließlich LAI, ALIA, Ccab und Cw aus hyperspektralen Daten abgeleitet werden. Künstliche neuronale Netze dienen dabei als Blackbox-Modelle, die in kurzer Zeit große Datenmengen verarbeiten können und somit einen entscheidenden Beitrag zur modernen angewandten Fernerkundung für eine breite User-Community leisten.After years of scientific and technical preparation, the launch of an unmanned German space-mission is planned to be initiated in 2020. The Environmental Mapping and Analysis Program (EnMAP) is going to provide an equally named hyperspectral imager to map land surfaces. Scientists of environmental disciplines of monitoring of ecosystems, agricultural, forestry and urban areas as well as coastal and inland waters, geology and soils prepared themselves for the upcoming data prior to the actual launch. Although there already exists a variety of useful algorithms for a profound analysis of spectral data, new challenges will arise given the uniqueness of the EnMAP-mission in the global context of remote sensing; i.e. coverage of the full range of the optical spectrum (420 nm – 2450 nm) in combination with a moderate spatial resolution of 30 m and a high signal-to-noise ratio of at least 180 in the shortwave infrared and above 400 in the visible spectrum. This enables an imaging quality which to this date has only been reached by airborne systems. The efforts of this dissertation comprise activities in the scientific preparation phase for agro-geographical tasks. Algorithms and tools for an analysis of hyperspectral data are being provided for free in the QGIS-plugin EnMAP-Box 3. Urgent questions in the agricultural sector revolve around the derivation of biochemical and biophysical parameters from remote sensing data. For this reason, the overarching objective of this promotion is the development of a scientific EnMAP-tool for managed areas of vegetation (EnMAP Managed Vegetation Scientific Processor). At first, an extensive field campaign was planned and then started in April, 2014. Apart from spectral observations of leaves, canopies and soils in a winter wheat and a maize field, also relevant plant parameters were acquired at the exact same spots. Namely, they are the Leaf Area Index (LAI), leaf chlorophyll content (Ccab), leaf water content (EWT or Cw), relative dry leaf weight (LMA or Cm), Average Leaf Inclination Angle (ALIA) as well as other secondary parameters like canopy height, phenological stage and the solar vector. Spectral measurements were captured from different observation angles to match ground data with the sensing geometry of the future EnMAP-satellite, which can be tilted up to 30° orthogonal to its direction of flight. A common procedure to derive relevant crop parameters is to make use of the radiative transfer model PROSAIL, which simulates the spectral signal of a vegetated surface based on biophysical and biochemical input parameters. If this process is reverted, said parameters can be derived from measured spectral data. To do so, a Look-Up-Table (LUT) is built containing model runs of PROSAIL and then subsequently compared against spectra from the field campaigns. With this approach of LUT-inversions from different observation angles, an accuracy of 18 % could be achieved for LAI and 20 % for Ccab. Strong anisotropic effects, i.e. dependence on illumination geometry and sensor orientation, were identified for winter wheat mainly in the early stages of plant development. In a consecutive study about uncertainties of the spectral model, PROSAIL results fed with in situ measured crop parameters as input, were opposed to their associated reflectance signatures. A strong deviation between measured and modelled spectra was observed, which – in the case of winter wheat – showed a seasonal behavior. The model tended to overestimate reflectances in the near infrared for early phenological stages and to underestimate them at end of the growing period. The parametrization of the model was identified as an uncertainty factor if the ALIA parameter is interpreted as true physical leaf inclinations. It was concluded that a separation of LAI and ALIA at inversion of PROSAIL prevents an adequate estimation of the less sensitive parameters. The development of the vegetation processor required the use of Machine Learning Regression Algorithms (MLRA), since distribution of large LUTs to the user would be impracticable. The MLRAs were trained with synthetic datasets with primary importance to optimize their hyperparameters, before attempting to apply the algorithms to real spectral data. Significant results could not be obtained until training data were altered with artificial noise, because algorithms suffered from overfitting to the model environment. Executing the processor allowed to derive LAI, ALIA, Ccab and Cw from hyperspectral data. Artificial neural networks served as black box models, which digest great amount of data in a short period of time and thus make a decisive contribution to modern applied remote sensing with relevance for a broad user-community

Aprendizado ativo baseado em atributos contextuais de superpixel para classificação de imagem de sensoriamento remoto

Orientadores: Alexandre Xavier Falcão, Jefersson Alex dos SantosDissertação (mestrado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Recentemente, técnicas de aprendizado de máquina têm sido propostas para criar mapas temáticos a partir de imagens de sensoriamento remoto. Estas técnicas podem ser divididas em métodos de classificação baseados em pixels ou regiões. Este trabalho concentra-se na segunda abordagem, uma vez que estamos interessados em imagens com milhões de pixels e a segmentação da imagem em regiões (superpixels) pode reduzir consideravelmente o número de amostras a serem classificadas. Porém, mesmo utilizando superpixels, o número de amostras ainda é grande para anotá-las manualmente e treinar o classificador. As técnicas de aprendizado ativo propostas resolvem este problema começando pela seleção de um conjunto pequeno de amostras selecionadas aleatoriamente. Tais amostras são anotadas manualmente e utilizadas para treinar a primeira instância do classificador. Em cada iteração do ciclo de aprendizagem, o classificador atribui rótulos e seleciona as amostras mais informativas para a correção/confirmação pelo usuário, aumentando o tamanho do conjunto de treinamento. A instância do classificador é melhorada no final de cada iteração pelo seu treinamento e utilizada na iteração seguinte até que o usuário esteja satisfeito com o classificador. Observamos que a maior parte dos métodos reclassificam o conjunto inteiro de dados em cada iteração do ciclo de aprendizagem, tornando este processo inviável para interação com o usuário. Portanto, enderaçamos dois problemas importantes em classificação baseada em regiões de imagens de sensoriamento remoto: (a) a descrição efetiva de superpixels e (b) a redução do tempo requerido para seleção de amostras em aprendizado ativo. Primeiro, propusemos um descritor contextual de superpixels baseado na técnica de sacola de palavras, que melhora o resultado de descritores de cor e textura amplamente utilizados. Posteriormente, propusemos um método supervisionado de redução do conjunto de dados que é baseado em um método do estado da arte em aprendizado ativo chamado Multi-Class Level Uncertainty (MCLU). Nosso método mostrou-se tão eficaz quanto o MCLU e ao mesmo tempo consideravelmente mais eficiente. Adicionalmente, melhoramos seu desempenho por meio da aplicação de um processo de relaxação no mapa de classificação, utilizando Campos Aleatórios de MarkovAbstract: In recent years, machine learning techniques have been proposed to create classification maps from remote sensing images. These techniques can be divided into pixel- and region-based image classification methods. This work concentrates on the second approach, since we are interested in images with millions of pixels and the segmentation of the image into regions (superpixels) can considerably reduce the number of samples for classification. However, even using superpixels the number of samples is still large for manual annotation of samples to train the classifier. Active learning techniques have been proposed to address the problem by starting from a small set of randomly selected samples, which are manually labeled and used to train a first instance of the classifier. At each learning iteration, the classifier assigns labels and selects the most informative samples for user correction/confirmation, increasing the size of the training set. An improved instance of the classifier is created by training, after each iteration, and used in the next iteration until the user is satisfied with the classifier. We observed that most methods reclassify the entire pool of unlabeled samples at every learning iteration, making the process unfeasible for user interaction. Therefore, we address two important problems in region-based classification of remote sensing images: (a) the effective superpixel description and (b) the reduction of the time required for sample selection in active learning. First, we propose a contextual superpixel descriptor, based on bag of visual words, that outperforms widely used color and texture descriptors. Second, we propose a supervised method for dataset reduction that is based on a state-of-art active learning technique, called Multi-Class Level Uncertainty (MCLU). Our method has shown to be as effective as MCLU, while being considerably more efficient. Additionally, we further improve its performance by applying a relaxation process on the classification map by using Markov Random FieldsMestradoCiência da ComputaçãoMestre em Ciência da Computaçã

Hyperspectral Imaging for Fine to Medium Scale Applications in Environmental Sciences

The aim of the Special Issue “Hyperspectral Imaging for Fine to Medium Scale Applications in Environmental Sciences” was to present a selection of innovative studies using hyperspectral imaging (HSI) in different thematic fields. This intention reflects the technical developments in the last three decades, which have brought the capacity of HSI to provide spectrally, spatially and temporally detailed data, favoured by e.g., hyperspectral snapshot technologies, miniaturized hyperspectral sensors and hyperspectral microscopy imaging. The present book comprises a suite of papers in various fields of environmental sciences—geology/mineral exploration, digital soil mapping, mapping and characterization of vegetation, and sensing of water bodies (including under-ice and underwater applications). In addition, there are two rather methodically/technically-oriented contributions dealing with the optimized processing of UAV data and on the design and test of a multi-channel optical receiver for ground-based applications. All in all, this compilation documents that HSI is a multi-faceted research topic and will remain so in the future

Framework for Automatic Identification of Paper Watermarks with Chain Codes

Title from PDF of title page viewed May 21, 2018Dissertation advisor: Reza DerakhshaniVitaIncludes bibliographical references (pages 220-235)Thesis (Ph.D.)--School of Computing and Engineering. University of Missouri--Kansas City, 2017In this dissertation, I present a new framework for automated description, archiving, and identification of paper watermarks found in historical documents and manuscripts. The early manufacturers of paper have introduced the embedding of identifying marks and patterns as a sign of a distinct origin and perhaps as a signature of quality. Thousands of watermarks have been studied, classified, and archived. Most of the classification categories are based on image similarity and are searchable based on a set of defined contextual descriptors. The novel method presented here is for automatic classification, identification (matching) and retrieval of watermark images based on chain code descriptors (CC). The approach for generation of unique CC includes a novel image preprocessing method to provide a solution for rotation and scale invariant representation of watermarks. The unique codes are truly reversible, providing high ratio lossless compression, fast searching, and image matching. The development of a novel distance measure for CC comparison is also presented. Examples for the complete process are given using the recently acquired watermarks digitized with hyper-spectral imaging of Summa Theologica, the work of Antonino Pierozzi (1389 – 1459). The performance of the algorithm on large datasets is demonstrated using watermarks datasets from well-known library catalogue collections.Introduction -- Paper and paper watermarks -- Automatic identification of paper watermarks -- Rotation, Scale and translation invariant chain code -- Comparison of RST_Invariant chain code -- Automatic identification of watermarks with chain codes -- Watermark composite feature vector -- Summary -- Appendix A. Watermarks from the Bernstein Collection used in this study -- Appendix B. The original and transformed images of watermarks -- Appendix C. The transformed and scaled images of watermarks -- Appendix D. Example of chain cod

The brushstroke and materials of Amadeo de Souza-Cardoso combined in an authentication tool

Nowadays, authentication studies for paintings require a multidisciplinary approach, based on the contribution of visual features analysis but also on characterizations of materials and techniques. Moreover, it is important that the assessment of the authorship of a painting is supported by technical studies of a selected number of original artworks that cover the entire career of an artist. This dissertation is concerned about the work of modernist painter Amadeo de Souza-Cardoso. It is divided in three parts. In the first part, we propose a tool based on image processing that combines information obtained by brushstroke and materials analysis. The resulting tool provides qualitative and quantitative evaluation of the authorship of the paintings; the quantitative element is particularly relevant, as it could be crucial in solving authorship controversies, such as judicial disputes. The brushstroke analysis was performed by combining two algorithms for feature detection, namely Gabor filter and Scale Invariant Feature Transform. Thanks to this combination (and to the use of the Bag-of-Features model), the proposed method shows an accuracy higher than 90% in distinguishing between images of Amadeo’s paintings and images of artworks by other contemporary artists. For the molecular analysis, we implemented a semi-automatic system that uses hyperspectral imaging and elemental analysis. The system provides as output an image that depicts the mapping of the pigments present, together with the areas made using materials not coherent with Amadeo’s palette, if any. This visual output is a simple and effective way of assessing the results of the system. The tool proposed based on the combination of brushstroke and molecular information was tested in twelve paintings obtaining promising results. The second part of the thesis presents a systematic study of four selected paintings made by Amadeo in 1917. Although untitled, three of these paintings are commonly known as BRUT, Entrada and Coty; they are considered as his most successful and genuine works. The materials and techniques of these artworks have never been studied before. The paintings were studied with a multi-analytical approach using micro-Energy Dispersive X-ray Fluorescence spectroscopy, micro-Infrared and Raman Spectroscopy, micro-Spectrofluorimetry and Scanning Electron Microscopy. The characterization of Amadeo’s materials and techniques used on his last paintings, as well as the investigation of some of the conservation problems that affect these paintings, is essential to enrich the knowledge on this artist. Moreover, the study of the materials in the four paintings reveals commonalities between the paintings BRUT and Entrada. This observation is supported also by the analysis of the elements present in a photograph of a collage (conserved at the Art Library of the Calouste Gulbenkian Foundation), the only remaining evidence of a supposed maquete of these paintings. The final part of the thesis describes the application of the image processing tools developed in the first part of the thesis on a set of case studies; this experience demonstrates the potential of the tool to support painting analysis and authentication studies. The brushstroke analysis was used as additional analysis on the evaluation process of four paintings attributed to Amadeo, and the system based on hyperspectral analysis was applied on the painting dated 1917. The case studies therefore serve as a bridge between the first two parts of the dissertation

Computer Vision System for Non-Destructive and Contactless Evaluation of Quality Traits in Fresh Rocket Leaves (Diplotaxis Tenuifolia L.)

La tesi di dottorato è incentrata sull'analisi di tecnologie non distruttive per il controllo della qualità dei prodotti agroalimentari, lungo l'intera filiera agroalimentare. In particolare, la tesi riguarda l'applicazione del sistema di visione artificiale per valutare la qualità delle foglie di rucola fresh-cut. La tesi è strutturata in tre parti (introduzione, applicazioni sperimentali e conclusioni) e in cinque capitoli, rispettivamente il primo e il secondo incentrati sulle tecnologie non distruttive e in particolare sui sistemi di computer vision per il monitoraggio della qualità dei prodotti agroalimentari. Il terzo, quarto e quinto capitolo mirano a valutare le foglie di rucola sulla base della stima di parametri qualitativi, considerando diversi aspetti: (i) la variabilità dovuta alle diverse pratiche agricole, (ii) la senescenza dei prodotti confezionati e non, e (iii) lo sviluppo e sfruttamento dei vantaggi di nuovi modelli più semplici rispetto al machine learning utilizzato negli esperimenti precedenti. Il lavoro di ricerca di questa tesi di dottorato è stato svolto dall'Università di Foggia, dall'Istituto di Scienze delle Produzioni Alimentari (ISPA) e dall'Istituto di Tecnologie e Sistemi Industriali Intelligenti per le Manifatture Avanzate (STIIMA) del Consiglio Nazionale delle Ricerche (CNR). L’attività di ricerca è stata condotta nell'ambito del Progetto SUS&LOW (Sustaining Low-impact Practices in Horticulture through Non-destructive Approach to Provide More Information on Fresh Produce History & Quality), finanziato dal MUR-PRIN 2017, e volto a sostenere la qualità della produzione e dell'ambiente utilizzando pratiche agricole a basso input e la valutazione non distruttiva della qualità di prodotti ortofrutticoli.The doctoral thesis focused on the analysis of non-destructive technologies available for the control quality of agri-food products, along the whole supply chain. In particular, the thesis concerns the application of computer vision system to evaluate the quality of fresh rocket leaves. The thesis is structured in three parts (introduction, experimental applications and conclusions) and in 5 chapters, the first and second focused on non-destructive technologies and in particular on computer vision systems for monitoring the quality of agri-food products, respectively. The third, quarter, and fifth chapters aim to assess the rocket leaves based on the estimation of quality aspects, considering different aspects: (i) the variability due to the different agricultural practices, (ii) the senescence of packed and unpacked products, and (iii) development and exploitation of the advantages of new models simpler than the machine learning used in the previous experiments. The research work of this doctoral thesis was carried out by the University of Foggia, the Institute of Science of Food Production (ISPA) and the Institute of Intelligent Industrial Technologies and Systems for Advanced Manufacturing (STIIMA) of National Research Council (CNR). It was conducted within the Project SUS&LOW (Sustaining Low-impact Practices in Horticulture through Non-destructive Approach to Provide More Information on Fresh Produce History & Quality), funded by MUR- PRIN 2017, and aimed at sustaining quality of production and of the environment using low input agricultural practices and non-destructive quality evaluation