Hyperspectral benthic mapping from underwater robotic platforms

We live on a planet of vast oceans; 70% of the Earth's surface is covered in water. They are integral to supporting life, providing 99% of the inhabitable space on Earth. Our oceans and the habitats within them are under threat due to a variety of factors. To understand the impacts and possible solutions, the monitoring of marine habitats is critically important. Optical imaging as a method for monitoring can provide a vast array of information however imaging through water is complex. To compensate for the selective attenuation of light in water, this thesis presents a novel light propagation model and illustrates how it can improve optical imaging performance. An in-situ hyperspectral system is designed which comprised of two upward looking spectrometers at different positions in the water column. The downwelling light in the water column is continuously sampled by the system which allows for the generation of a dynamic water model. In addition to the two upward looking spectrometers the in-situ system contains an imaging module which can be used for imaging of the seafloor. It consists of a hyperspectral sensor and a trichromatic stereo camera. New calibration methods are presented for the spatial and spectral co-registration of the two optical sensors. The water model is used to create image data which is invariant to the changing optical properties of the water and changing environmental conditions. In this thesis the in-situ optical system is mounted onboard an Autonomous Underwater Vehicle. Data from the imaging module is also used to classify seafloor materials. The classified seafloor patches are integrated into a high resolution 3D benthic map of the surveyed site. Given the limited imaging resolution of the hyperspectral sensor used in this work, a new method is also presented that uses information from the co-registered colour images to inform a new spectral unmixing method to resolve subpixel materials

Probabilistic Reconstruction of Color for Species’ Classification Underwater

Color is probably the most informative cue for object recognition and classification in natural scenes. Difference in shades can indicate to the biologist the potential for diversity of species or stress on the habitats. However, severe color distortions may occur in underwater imagery due to wavelength-dependent attenuation of light. Affordable tri-chromatic sensors are used to record the ambient light condition and color correct the imagery, but results show that this approach works reliably only under highly controllable conditions. This paper proposes an approach that combines hyperspectral data collected for the object of interest, hardware properties of the imaging sensor, and exterior conditions (optical properties of water and illumination) with tri-chromatic underwater imagery. Due to ambiguity of color reconstruction underwater, demonstrated in the paper, a probabilistic approach is used for classification that allows the identification of the object of interest from other objects

A computational approach to the quantification of animal camouflage

Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy at the Massachusetts Institute of Technology and the Woods Hole Oceanographic Institution June 2014Evolutionary pressures have led to some astonishing camouflage strategies in the animal kingdom. Cephalopods like cuttlefish and octopus mastered a rather unique skill: they can rapidly adapt the way their skin looks in color, texture and pattern, blending in with their backgrounds. Showing a general resemblance to a visual background is one of the many camouflage strategies used in nature. For animals like cuttlefish that can dynamically change the way they look, we would like to be able to determine which camouflage strategy a given pattern serves. For example, does an inexact match to a particular background mean the animal has physiological limitations to the patterns it can show, or is it employing a different camouflage strategy (e.g., disrupting its outline)? This thesis uses a computational and data-driven approach to quantify camouflage patterns of cuttlefish in terms of color and pattern. First, we assess the color match of cuttlefish to the features in its natural background in the eyes of its predators. Then, we study overall body patterns to discover relationships and limitations between chromatic components. To facilitate repeatability of our work by others, we also explore ways for unbiased data acquisition using consumer cameras and conventional spectrometers, which are optical imaging instruments most commonly used in studies of animal coloration and camouflage. This thesis makes the following contributions: (1) Proposes a methodology for scene-specific color calibration for the use of RGB cameras for accurate and consistent data acquisition. (2) Introduces an equation relating the numerical aperture and diameter of the optical fiber of a spectrometer to measurement distance and angle, quantifying the degree of spectral contamination. (3) Presents the first study assessing the color match of cuttlefish (S. officinalis) to its background using in situ spectrometry. (4) Develops a computational approach to pattern quantification using techniques from computer vision, image processing, statistics and pattern recognition; and introduces Cuttlefish72x5, the first database of calibrated raw (linear) images of cuttlefish.Funding was provided by the National Science Foundation, Office of Naval Research, NIH-NEI, and the Woods Hole Oceanographic Institution Academic Programs Office

Challenges in video based object detection in maritime scenario using computer vision

This paper discusses the technical challenges in maritime image processing and machine vision problems for video streams generated by cameras. Even well documented problems of horizon detection and registration of frames in a video are very challenging in maritime scenarios. More advanced problems of background subtraction and object detection in video streams are very challenging. Challenges arising from the dynamic nature of the background, unavailability of static cues, presence of small objects at distant backgrounds, illumination effects, all contribute to the challenges as discussed here

Hyperspectral Data Acquisition and Its Application for Face Recognition

Current face recognition systems are rife with serious challenges in uncontrolled conditions: e.g., unrestrained lighting, pose variations, accessories, etc. Hyperspectral imaging (HI) is typically employed to counter many of those challenges, by incorporating the spectral information within different bands. Although numerous methods based on hyperspectral imaging have been developed for face recognition with promising results, three fundamental challenges remain: 1) low signal to noise ratios and low intensity values in the bands of the hyperspectral image specifically near blue bands; 2) high dimensionality of hyperspectral data; and 3) inter-band misalignment (IBM) correlated with subject motion during data acquisition. This dissertation concentrates mainly on addressing the aforementioned challenges in HI. First, to address low quality of the bands of the hyperspectral image, we utilize a custom light source that has more radiant power at shorter wavelengths and properly adjust camera exposure times corresponding to lower transmittance of the filter and lower radiant power of our light source. Second, the high dimensionality of spectral data imposes limitations on numerical analysis. As such, there is an emerging demand for robust data compression techniques with lows of less relevant information to manage real spectral data. To cope with these challenging problems, we describe a reduced-order data modeling technique based on local proper orthogonal decomposition in order to compute low-dimensional models by projecting high-dimensional clusters onto subspaces spanned by local reduced-order bases. Third, we investigate 11 leading alignment approaches to address IBM correlated with subject motion during data acquisition. To overcome the limitations of the considered alignment approaches, we propose an accurate alignment approach ( A3) by incorporating the strengths of point correspondence and a low-rank model. In addition, we develop two qualitative prediction models to assess the alignment quality of hyperspectral images in determining improved alignment among the conducted alignment approaches. Finally, we show that the proposed alignment approach leads to promising improvement on face recognition performance of a probabilistic linear discriminant analysis approach

ON THE LOGIC, METHOD AND SCIENTIFIC DIVERSITY OF TECHNICAL SYSTEMS: AN INQUIRY INTO THE DIAGNOSTIC MEASUREMENT OF HUMAN SKIN

This dissertation explores some of the scientific, technical and cultural history of human skin measurement and diagnostics. Through a significant collection of primary texts and case studies, I track the changing technologies and methods used to measure skin, as well as the scientific and sociotechnical applications. I then map these histories onto some of the diverse understandings of the human body, physics, biology, natural philosophy and language that underpinned the scientific enterprise of skin measurement. The main argument of my thesis demonstrates how these diverse histories of science historically and theoretically inform the succeeding methods and applications for skin measurement from early Greek medicine, to beginnings of Anthropology as scientific discipline, to the emergence of scientific racism, to the age of digital imaging analysis, remote sensing, algorithms, massive databases and biometric technologies; further, these new digital applications go beyond just health diagnostics and are creating new technical categorizations of human skin divorced from the established ethical mechanisms of modern science. Based on this research, I inquire how communication practices within the scientific enterprise address the ethical and historical implications for a growing set of digital biometric applications with industrial, military, sociopolitical and public functions

Retrieval of Lake Erie Water Quality Parameters from Satellite Remote Sensing and Impact on Simulations with a 1-D Lake Model

Lake Erie is a freshwater lake, and the most southern of the Laurentian Great Lakes in North America. It is the smallest by volume, the fourth largest in surface area (25,700 km2), and the shallowest of the Laurentian Great Lakes. The lake’s high productivity and warm weather in its watershed has attracted one-third of the total human population of the Great Lake’s basin. The industrial and agricultural activities of this huge population has caused serious environmental problems for Lake Erie namely harmful algal blooms, dissolved organic/inorganic matters from river inputs, and sediment loadings. If these sorts of water contaminations exceed a certain level, it can seriously influence the lake ecosystem. Hence, an effective and continuous water quality monitoring program is of outmost importance for Lake Erie. The use of Earth observation satellites to improve monitoring of environmental changes in water bodies has been receiving increased attention in recent years. Satellite observations can provide long term spatial and temporal trends of water quality indicators which cannot be achieved through discontinuous conventional point-wise in situ sampling. Different regression-based empirical models have been developed in the literature to derive the water optical properties from a single (or band ratio of) remote sensing reflectance (radiance). In situ measurements are used to build these regressions. The repeated in situ measurements in space and/or time causes clustered and correlated data that violates the assumption of regression models. Considering this correlation in developing regression models was one of the topics examined in this thesis. More complicated semi-analytical models are applied in Case II waters, aiming to distinguish several constituents confounding water-leaving signals more effectively. The MERIS neural network (NN) algorithms are the most widely used among semi-analytical models. The applicability of these algorithms to derive chl-a concentration and Secchi Disk Depth (SDD) in Lake Erie was assessed for the first time in this thesis. Satellite-observations of water turbidity were then coupled with a 1-D lake model to improve its performance on Lake Erie, where the common practice is to use a constant value for water turbidity in the model due to insufficient in situ measurements of water turbidity for lakes globally. In the first chapter, four well-established MERIS NN algorithms to derive chl-a concentration as well as two band-ratio chl-a related indices were evaluated against in situ measurements. The investigated products are those produced by NN algorithms, including Case 2 Regional (C2R), Eutrophic (EU), Free University of Berlin WeW WATER processor (FUB/WeW), and CoastColour (CC) processors, as well as from band-ratio algorithms of fluorescence line height (FLH) and maximum chlorophyll index (MCI). Two approaches were taken to compare and evaluate the performance of these algorithms to predict chl-a concentration after lake-specific calibration of the algorithms. First, all available chl-a matchups, which were collected from different locations on the lake, were evaluated at once. In the second approach, a classification of three optical water types was applied, and the algorithms’ performance was assessed for each type, individually. The results of this chapter show that the FUB/WeW processor outperforms other algorithms when the full matchup data of the lake was used (root mean square error (RMSE) = 1.99 mg m-3, index-of-agreement (I_a) = 0.67). However, the best performing algorithm was different when each water optical type was investigated individually. The findings of this study provide practical and valuable information on the effectiveness of the already existing MERIS-based algorithms to derive the trophic state of Lake Erie, an optically complex lake. Unlike the first chapter, where physically-based and already trained algorithms were implemented to evaluate satellite derived chl-a concentration, in the next chapter, two lake-specific, robust semi-empirical algorithms were developed to derive chl-a and SDD using Linear Mixed Effect (LME) models. LME considers the correlation that exists in the field measurements which have been repeatedly performed in space and time. Each developed algorithm was then employed to investigate the monthly-averaged spatial and temporal trends of chl-a concentration and water turbidity during the period of 2005-2011. SDD was used as the indicator of water turbidity. LME models were developed between the logarithmic scale of the parameters and the band ratio of B7:665 nm to B9:708.75 nm for log10chl-a, and the band ratio of B6:620 nm to B4:510 nm for log10SDD. The models resulted in RMSE of 0.30 for log10chl-a and 0.19 for log10SDD. Maps produced with the two LME models revealed distinct monthly patterns for different regions of the lake that are in agreement with the biogeochemical properties of Lake Erie. Lastly the water turbidity (extinction coefficient; Kd) of Lake Erie was estimated using the globally available satellite-based CC product. The CC-derived Kd product was in a good agreement with the SDD field observations (RMSE=0.74 m-1, mean bias error (MBE)=0.53 m-1, I_a=0.53). CC-derived Kd was then used as input for simulations with the 1-D Freshwater Lake (FLake) model. An annual average constant Kd value calculated from the CC product improved simulation results of lake surface water temperature (LSWT) compared to a “generic” constant value (0.2 m-1) used in previous studies (CC lake-specific yearly average Kd value: RMSE=1.54 ºC, MBE= -0.08 ºC; generic constant Kd value: RMSE=1.76 ºC, MBE= -1.26 ºC). Results suggest that a time-independent, lake-specific, and constant Kd value from CC can improve FLake LSWT simulations with sufficient accuracy. A sensitivity analysis was also conducted to assess the performance of FLake to simulate LSWT, mean water column temperature (MWCT) and mixed layer depth (MLD) using different values of Kd. Results showed that the model is very sensitive to the variations of Kd, particularly when Kd value is below 0.5 m-1. The sensitivity of FLake to Kd variations was more pronounced in simulations of MWCT and MLD. This study shows that a global mapping of the extinction coefficient can be created using satellite-based observations of lakes optical properties to improve the 1-D FLake model. Overall, results from this thesis clearly demonstrate the benefits of remote sensing measurements of water quality parameters (such as chl-a concentration and water turbidity) for lake monitoring. Also, this research shows that the integration of space-borne water clarity (extinction coefficient) measurements into the 1-D FLake model improves simulations of LSWT

A quantitative and qualitative approach to cuttlefish (Sepia officinalis) body patterning

Cuttlefish are renowned for their ability to quickly alter the colour and texture of their skin, for camouflage and communication. This is due to the presence of thousands of pigment-filled sacs, known as chromatophores, which are distributed across the skin. The chromatophores are innervated by motoneurons, which dilate the chromatophores to create the spots, stripes, and other markings, known as chromatic components. There are 34 recognized chromatic components, and it is an interesting question how cuttlefish coordinate the expression of these components to camouflage and communicate. The digital age has introduced new, powerful algorithms and methods to tease out subtle features in the coloration patterns, by means of image registration, segmentation, and identification, as well as methods for modeling the underlying control systems. These tools offer major new insights into the mechanisms of visual perception. In addition, powerful techniques have recently been developed that have yet to be applied to this complex visual motor control system. These methods have large potential in helping discover what features between the pattern and the environment are necessary to prevent detection. Here I present four laboratory experiments, that for the first time use machine learning models, to investigate cuttlefish pattern formation, implementation, and information. The first two experimental chapters investigate how cuttlefish orchestrate their chromatic components for camouflage patterns, and what strategies they employ on diverse backgrounds. I demonstrate that components are expressed more independently than previously believed, finding that the range of patterns expressed lie on a continuum, allowing us to suggest a revised classification scheme for cuttlefish body patterns. The diversity of patterns seem to imply that a cuttlefish could use its repertoire flexibly to display the maximally cryptic pattern for a given background, however I show that cuttlefish to not in fact select a single (possibly optimal) camouflage pattern, continually alter their appearance on a given background, and that the frequency of change increases in relation to the size of the objects in the environment. My third chapter investigated the language-like properties of cuttlefish communication using human speech recognition models. From our subset of cuttlefish patterns, I discovered cuttlefish utilize a lexicon of 10 patterns, with language-like properties such as: they obeyed Zipf’s law, contained around 1.6 bits per display, and interestingly, while 2 patterns were visually similar, they were displayed in separate contexts. By implementing a regression onto the patterns, I introduce a basic dictionary of cuttlefish terms and their meaning. From my investigations into cuttlefish intraspecific signaling, I discovered two previously undocumented patterns, used in agonistic encounters between cuttlefish. My final chapter describes these patterns and the contexts they are displayed

Going hyperspectral: the 'unseen' captured?

All objects, name them soil, water, trees, vegetation, structures, metals, paints or fabrics, create a unique spectral fingerprint. A sensor determines these fingerprints by measuring reflected light, most of which registers in wavelengths, or bands, invisible to humans. This is what the crime scene investigation (CSI) television programs have popularized how DNA or fingerprints can be used to solve crimes. Similarly, forest CSI of “seeing” the trees in the deep high mountain tropical forest is now a major focus in the air and spaceborne hyperspectral sensing technology and in other different applications such as agriculture, environment, geology, transportation, security, and several others. The availability of sub-meter resolution colour imagery from satellites coupled with internet based services like Google Earth and Microsoft Virtual Earth have resulted in an enormous interest in remote sensing among the general public. The ability to see one’s home or familiar landmarks in an image taken from hundreds of kilometers above the earth elicits wonder and awe. Deciding where, when, what and how to sense or measure the DNA of individual trees from the air or space is a crucial question in the sustainable development and management of our Malaysian tropical forest ecosystems. However, to monitor, quantify, map and understand the content and nature of our forest, one would ideally like to monitor it everywhere and all the time too. This is impossible, and consequently, forest engineers must select relatively very high to high near to real time resolution sensors with the ability to transcend boundaries, capabilities, features and interfacing realms for such measurement. The dynamic interplay of these elements is precisely coordinated by signaling networks that orchestrate their interactions. High-throughput experimental and analytical techniques now provide forest engineers with incredibly rich and potentially revealing datasets from both air and spaceborne hyperspectral sensors (also known as imaging spectrometers). However, it is impossible to exhaustively explore the full experimental and operational hyperspectral sensors available in the market out there and so forest engineers must judiciously choose which one is the best to perform and fulfill their project objectives and missions. The complexity and high-dimensionality of these systems makes it incredibly difficult for forest engineers and other users alone to manage and optimize sensing processes. In order to add or derive value from a hyperspectral remotely sensed image several factors such as resolution, swath, and signal to noise ratio, amongst others need to be considered. A grand challenge for the forest engineer’s scientific discovery in the 21st Century is therefore, to devise very high real-time ultra-spatial and spectral air and space borne sensors that automatically measure and adapt sensing operations in large-scale and economical systems with the unseen captured. This lecture therefore focuses on the emerging theory, origin of the hyperspectral sensors, research, practice, limitations and identifies future challenge and outlook of hyperspectral sensing systems in the quest towards a sustainable Malaysian forestry context and other different applications to capture the “unseen”. It is quite certain that advances in hyperspectral remote sensing and more sophisticated analytical methods will resolve any “unseen” issues in time with the best approach of transcending boundaries and interfacing remote sensing data with precise information from the field plots. Unfortunately, as a relatively new analytical technique, the full potential of air and spaceborne hyperspectral imaging has not yet been realized in Malaysi