Towards a Generalized Machine Learning Approach for Estimating Chlorophyll Values in Inland Waters with Spectral Data

Wasser ist ein wesentliches Element des Lebens. Seine Qualität ist jedoch bedroht, zum Beispiel durch schädliche Algenblüten oder anthropogene Verschmutzungen. Regelmäßige Kontrollen ermöglichen das Erkennen von Veränderungen der Wasserqualität von Binnengewässern. Konventionelle Wasserqualitätskontrollen werden hauptsächlich mittels In-situ-Probenahmen durchgeführt, eine teure und arbeitsintensive Vorgehensweise. Spektrale Fernerkundung kann eine Alternative zu In-situ-Beprobungen sein. Die sichtbare und nahinfrarote Strahlung, die von einem Sensor aufgenommen wird, hat mit dem Wasserkörper und dessen Inhaltsstoffen interagiert. Dadurch enthält die Strahlung Informationen über Absorptions- und Streuprozesse in der Wassersäule. Ein Parameter, der stark mit der Strahlung wechselwirkt, ist das pflanzliche Pigment Chlorophyll a. Chlorophyll a ist ein Proxy für die Phytoplanktonabundanz und kann daher mit der Wasserqualität in Verbindung gebracht werden. Die spektrale Überlappung mit anderen Wasserinhaltsstoffen erschwert die Bestimmung des Chlorophyll a-Gehalts mit spektralen Daten in der Wassersäule. Daher ist ein zuverlässiger Modellierungsansatz erforderlich, um diese nicht-lineare Regressionsaufgabe zu lösen und damit kontinuierliche Chlorophyll a-Werte aus Spektraldaten zu gewinnen. Eine zusätzliche Anforderung an einen solchen Ansatz ist die Anwendbarkeit auf die meisten der weltweiten Binnengewässer, da der Mangel an Referenzdaten nicht erlaubt, spezialisierte Modelle für jeden einzelnen See zu generieren. Diese Generalisierungsanforderung passt perfekt zu Ansätzen des überwachten maschinellen Lernens. Ein Hauptziel dieser Arbeit ist daher das Trainieren und Evaluieren von überwachten maschinellen Lernverfahren zum Schätzen kontinuierlicher Chlorophyll a-Werte von mehreren Binnengewässern. Die untersuchten Studien stützen sich dabei vollständig auf spektrale In-situ-Messungen. Dieser Aufbau erlaubt eine detailliertere Analyse der Beziehungen zwischen spektralen Daten und Wasserparametern. Außerdem wird der Einfluss der Atmosphäre verringert. Drei verschiedene Datensätze wurden im Rahmen dieser Arbeit aufgenommen, um den Generalisierungsprozess der generierten Modelle zu untersuchen. Die Variabilität der Datensätze nimmt dabei sukzessive zu. Daher wurden für diese Datensätze drei Studienkonfigurationen entworfen, die sukzessive die Anforderung zur Generalisierung der Modelle erhöhen. In der ersten Konfiguration werden lediglich Modelle untersucht, die sich auf ein einzelnes Gewässer beziehen. Im Gegensatz dazu stützt sich die letzte Konfiguration auf einen vollständig simulierten Datensatz für den Trainingsprozess der Modelle, während deren Evaluierung auf einem völlig unabhängigen Datensatz mit elf verschiedenen Binnengewässern erfolgt. Die Idee hinter diesem Konzept ist, wenn die Modelle die Chlorophyll a-Werte der elf völlig unbekannten Binnengewässer schätzen können, werden sie vermutlich auch weltweit, die Werte ähnlicher Binnengewässer schätzen können. Ein eindimensionales CNN als Vertreter der Deep-Learning-Verfahren hat sich dabei als das Modell mit den besten Generaliserungseigenschaften bei zufriedenstellender Schätzgenauigkeit erwiesen. Ein weiteres Augenmerk wird auf die spektrale Auflösung gelegt. Eine Verringerung der spektralen Auflösung von hyperspektral auf multispektral ist mit einem Informationsverlust verbunden. Die Schätzungsergebnisse aus dem eindimensionalen CNN zeigen, dass eine hyperspektrale Auflösung für ein vollständig generalisierendes Modell notwendig ist. Eine multispektrale Auflösung ist jedoch ausreichend für weniger generalisierende Modelle. Diese Erkenntnisse sind wichtig um im Hinblick auf ein zukünftiges Forschungsvorhaben den Upscaling-Ansatz auf reale Satellitendaten zu realisieren und damit eine flächendeckende Überwachung der Wasserqualität zu verwirklichen

Deep Learning Approaches for Seagrass Detection in Multispectral Imagery

Seagrass forms the basis for critically important marine ecosystems. Seagrass is an important factor to balance marine ecological systems, and it is of great interest to monitor its distribution in different parts of the world. Remote sensing imagery is considered as an effective data modality based on which seagrass monitoring and quantification can be performed remotely. Traditionally, researchers utilized multispectral satellite images to map seagrass manually. Automatic machine learning techniques, especially deep learning algorithms, recently achieved state-of-the-art performances in many computer vision applications. This dissertation presents a set of deep learning models for seagrass detection in multispectral satellite images. It also introduces novel domain adaptation approaches to adapt the models for new locations and for temporal image series. In Chapter 3, I compare a deep capsule network (DCN) with a deep convolutional neural network (DCNN) for seagrass detection in high-resolution multispectral satellite images. These methods are tested on three satellite images in Florida coastal areas and obtain comparable performances. In addition, I also propose a few-shot deep learning strategy to transfer knowledge learned by DCN from one location to the others for seagrass detection. In Chapter 4, I develop a semi-supervised domain adaptation method to generalize a trained DCNN model to multiple locations for seagrass detection. First, the model utilizes a generative adversarial network (GAN) to align marginal distribution of data in the source domain to that in the target domain using unlabeled data from both domains. Second, it uses a few labeled samples from the target domain to align class-specific data distributions between the two. The model achieves the best results in 28 out of 36 scenarios as compared to other state-of-the-art domain adaptation methods. In Chapter 5, I develop a semantic segmentation method for seagrass detection in multispectral time-series images. First, I train a state-of-the-art image segmentation method using an active learning approach where I use the DCNN classifier in the loop. Then, I develop an unsupervised domain adaptation (UDA) algorithm to detect seagrass across temporal images. I also extend our unsupervised domain adaptation work for seagrass detection across locations. In Chapter 6, I present an automated bathymetry estimation model based on multispectral satellite images. Bathymetry refers to the depth of the ocean floor and contributes a predominant role in identifying marine species in seawater. Accurate bathymetry information of coastal areas will facilitate seagrass detection by reducing false positives because seagrass usually do not grow beyond a certain depth. However, bathymetry information of most parts of the world is obsolete or missing. Traditional bathymetry measurement systems require extensive labor efforts. I utilize an ensemble machine learning-based approach to estimate bathymetry based on a few in-situ sonar measurements and evaluate the proposed model in three coastal locations in Florida

Ocean remote sensing techniques and applications: a review (Part II)

As discussed in the first part of this review paper, Remote Sensing (RS) systems are great tools to study various oceanographic parameters. Part I of this study described different passive and active RS systems and six applications of RS in ocean studies, including Ocean Surface Wind (OSW), Ocean Surface Current (OSC), Ocean Wave Height (OWH), Sea Level (SL), Ocean Tide (OT), and Ship Detection (SD). In Part II, the remaining nine important applications of RS systems for ocean environments, including Iceberg, Sea Ice (SI), Sea Surface temperature (SST), Ocean Surface Salinity (OSS), Ocean Color (OC), Ocean Chlorophyll (OCh), Ocean Oil Spill (OOS), Underwater Ocean, and Fishery are comprehensively reviewed and discussed. For each application, the applicable RS systems, their advantages and disadvantages, various RS and Machine Learning (ML) techniques, and several case studies are discussed.Peer ReviewedPostprint (published version

Machine Learning for Classifying Marine Vegetation from Hyperspectral Drone Data in the Norwegian coast

Along the Norwegian coasts the presence of blue forests are the key marine habitats. Due to increased anthropogenic activity and climate change, the health and extent of the blue forests is threatened. However, no low-cost, reliable system for monitoring blue forests exists in Norway at this time. This thesis studied machine learning methods to classify marine vegetation from hyperspectral data acquired in Norway. The study area is situated by Larvik at Ølbergholmen. The dataset consists of 12 hyperspectral images with 173 spectral bands in the region 390 nm - 749 nm and corresponding labels of the different classes. This dataset was used to train and evaluate the machine learning methods. In addition, an independent dataset from a different site was used for robustness evaluation. Three machine learning methods were studied; Random Forest (RF), Support Vector Machines (SVM) and Convolutional Neural Network (CNN). The results indicate that the powerful CNN approach had the best performance during validation based on the computed statistical measures. However, when evaluated for robustness, RF performed the best. The computed confusion matrices for the validation and robustness studies revealed that the presence of a so-called turf algae caused difficulties in distinguishing between the classes, which is an important finding with regard to future research. This thesis has shown that machine learning can be used for monitoring blue forests and various marine vegetation species using hyperspectral drone imaging along the Norwegian coast

Remote Sensing of the Aquatic Environments

The book highlights recent research efforts in the monitoring of aquatic districts with remote sensing observations and proximal sensing technology integrated with laboratory measurements. Optical satellite imagery gathered at spatial resolutions down to few meters has been used for quantitative estimations of harmful algal bloom extent and Chl-a mapping, as well as winds and currents from SAR acquisitions. The knowledge and understanding gained from this book can be used for the sustainable management of bodies of water across our planet

Image sensors for wave monitoring in shore protection: Characterization through a machine learning algorithm

Waves propagating on the water surface can be considered as propagating in a dispersive medium, where gravity and surface tension at the air–water interface act as restoring forces. The velocity at which energy is transported in water waves is defined by the group velocity. The paper reports the use of video‐camera observations to study the impact of water waves on an urban shore. The video‐monitoring system consists of two separate cameras equipped with progressive RGB CMOS sensors that allow 1080p HDTV video recording. The sensing system delivers video signals that are processed by a machine learning technique. The scope of the research is to identify features of water waves that cannot be normally observed. First, a conventional modelling was performed using data delivered by image sensors together with additional data such as temperature, and wind speed, measured with dedicated sensors. Stealth waves are detected, as are the inverting phenomena encompassed in waves. This latter phenomenon can be detected only through machine learning. This double approach allows us to prevent extreme events that can take place in offshore and onshore areas

Sustainable marine ecosystems: deep learning for water quality assessment and forecasting

An appropriate management of the available resources within oceans and coastal regions is vital to guarantee their sustainable development and preservation, where water quality is a key element. Leveraging on a combination of cross-disciplinary technologies including Remote Sensing (RS), Internet of Things (IoT), Big Data, cloud computing, and Artificial Intelligence (AI) is essential to attain this aim. In this paper, we review methodologies and technologies for water quality assessment that contribute to a sustainable management of marine environments. Specifically, we focus on Deep Leaning (DL) strategies for water quality estimation and forecasting. The analyzed literature is classified depending on the type of task, scenario and architecture. Moreover, several applications including coastal management and aquaculture are surveyed. Finally, we discuss open issues still to be addressed and potential research lines where transfer learning, knowledge fusion, reinforcement learning, edge computing and decision-making policies are expected to be the main involved agents.Postprint (published version

Evaluation of Machine Learning Algorithms for Lake Ice Classification from Optical Remote Sensing Data

The topic of lake ice cover mapping from satellite remote sensing data has gained interest in recent years since it allows the extent of lake ice and the dynamics of ice phenology over large areas to be monitored. Mapping lake ice extent can record the loss of the perennial ice cover for lakes located in the High Arctic. Moreover, ice phenology dates, retrieved from lake ice maps, are useful for assessing long-term trends and variability in climate, particularly due to their sensitivity to changes in near-surface air temperature. However, existing knowledge-driven (threshold-based) retrieval algorithms for lake ice-water classification that use top-of-the-atmosphere (TOA) reflectance products do not perform well under the condition of large solar zenith angles, resulting in low TOA reflectance. Machine learning (ML) techniques have received considerable attention in the remote sensing field for the past several decades, but they have not yet been applied in lake ice classification from optical remote sensing imagery. Therefore, this research has evaluated the capability of ML classifiers to enhance lake ice mapping using multispectral optical remote sensing data (MODIS L1B (TOA) product). Chapter 3, the main manuscript of this thesis, presents an investigation of four ML classifiers (i.e. multinomial logistic regression, MLR; support vector machine, SVM; random forest, RF; gradient boosting trees, GBT) in lake ice classification. Results are reported using 17 lakes located in the Northern Hemisphere, which represent different characteristics regarding area, altitude, freezing frequency, and ice cover duration. According to the overall accuracy assessment using a random k-fold cross-validation (k = 100), all ML classifiers were able to produce classification accuracies above 94%, and RF and GBT provided above 98% classification accuracies. Moreover, the RF and GBT algorithms provided a more visually accurate depiction of lake ice cover under challenging conditions (i.e., high solar zenith angles, black ice, and thin cloud cover). The two tree-based classifiers were found to provide the most robust spatial transferability over the 17 lakes and performed consistently well across three ice seasons, better than the other classifiers. Moreover, RF was insensitive to the choice of the hyperparameters compared to the other three classifiers. The results demonstrate that RF and GBT provide a great potential to map accurately lake ice cover globally over a long time-series. Additionally, a case study applying a convolution neural network (CNN) model for ice classification in Great Slave Lake, Canada is presented in Appendix A. Eighteen images acquired during the the ice season of 2009-2010 were used in this study. The proposed CNN produced a 98.03% accuracy with the testing dataset; however, the accuracy dropped to 90.13% using an independent (out-of-sample) validation dataset. Results show the powerful learning performance of the proposed CNN with the testing data accuracy obtained. At the same time, the accuracy reduction of the validation dataset indicates the overfitting behavior of the proposed model. A follow-up investigation would be needed to improve its performance. This thesis investigated the capability of ML algorithms (both pixel-based and spatial-based) in lake ice classification from the MODIS L1B product. Overall, ML techniques showed promising performances for lake ice cover mapping from the optical remote sensing data. The tree-based classifiers (pixel-based) exhibited the potential to produce accurate lake ice classification at a large-scale over long time-series. In addition, more work would be of benefit for improving the application of CNN in lake ice cover mapping from optical remote sensing imagery

Deep Gaussian processes for biogeophysical parameter retrieval and model inversion

Parameter retrieval and model inversion are key problems in remote sensing and Earth observation. Currently, different approximations exist: a direct, yet costly, inversion of radiative transfer models (RTMs); the statistical inversion with in situ data that often results in problems with extrapolation outside the study area; and the most widely adopted hybrid modeling by which statistical models, mostly nonlinear and non-parametric machine learning algorithms, are applied to invert RTM simulations. We will focus on the latter. Among the different existing algorithms, in the last decade kernel based methods, and Gaussian Processes (GPs) in particular, have provided useful and informative solutions to such RTM inversion problems. This is in large part due to the confidence intervals they provide, and their predictive accuracy. However, RTMs are very complex, highly nonlinear, and typically hierarchical models, so that very often a single (shallow) GP model cannot capture complex feature relations for inversion. This motivates the use of deeper hierarchical architectures, while still preserving the desirable properties of GPs. This paper introduces the use of deep Gaussian Processes (DGPs) for bio-geo-physical model inversion. Unlike shallow GP models, DGPs account for complicated (modular, hierarchical) processes, provide an efficient solution that scales well to big datasets, and improve prediction accuracy over their single layer counterpart. In the experimental section, we provide empirical evidence of performance for the estimation of surface temperature and dew point temperature from infrared sounding data, as well as for the prediction of chlorophyll content, inorganic suspended matter, and coloured dissolved matter from multispectral data acquired by the Sentinel-3 OLCI sensor. The presented methodology allows for more expressive forms of GPs in big remote sensing model inversion problems.European Research Council (ERC) 647423Spanish Ministry of Economy and Competitiveness TIN2015-64210-R DPI2016-77869-C2-2-RSpanish Excellence Network TEC2016-81900-REDTLa Caixa Banking Foundation (Barcelona, Spain) 100010434 LCF-BQ-ES17-1160001