Automated classification of three-dimensional reconstructions of coral reefs using convolutional neural networks

© The Author(s), 2020. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Hopkinson, B. M., King, A. C., Owen, D. P., Johnson-Roberson, M., Long, M. H., & Bhandarkar, S. M. Automated classification of three-dimensional reconstructions of coral reefs using convolutional neural networks. PLoS One, 15(3), (2020): e0230671, doi: 10.1371/journal.pone.0230671.Coral reefs are biologically diverse and structurally complex ecosystems, which have been severally affected by human actions. Consequently, there is a need for rapid ecological assessment of coral reefs, but current approaches require time consuming manual analysis, either during a dive survey or on images collected during a survey. Reef structural complexity is essential for ecological function but is challenging to measure and often relegated to simple metrics such as rugosity. Recent advances in computer vision and machine learning offer the potential to alleviate some of these limitations. We developed an approach to automatically classify 3D reconstructions of reef sections and assessed the accuracy of this approach. 3D reconstructions of reef sections were generated using commercial Structure-from-Motion software with images extracted from video surveys. To generate a 3D classified map, locations on the 3D reconstruction were mapped back into the original images to extract multiple views of the location. Several approaches were tested to merge information from multiple views of a point into a single classification, all of which used convolutional neural networks to classify or extract features from the images, but differ in the strategy employed for merging information. Approaches to merging information entailed voting, probability averaging, and a learned neural-network layer. All approaches performed similarly achieving overall classification accuracies of ~96% and >90% accuracy on most classes. With this high classification accuracy, these approaches are suitable for many ecological applications.This study was funded by grants from the Alfred P. Sloan Foundation (BMH, BR2014-049; https://sloan.org), and the National Science Foundation (MHL, OCE-1657727; https://www.nsf.gov). The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript

Anatomical curve identification

Methods for capturing images in three dimensions are now widely available, with stereo-photogrammetry and laser scanning being two common approaches. In anatomical studies, a number of landmarks are usually identified manually from each of these images and these form the basis of subsequent statistical analysis. However, landmarks express only a very small proportion of the information available from the images. Anatomically defined curves have the advantage of providing a much richer expression of shape. This is explored in the context of identifying the boundary of breasts from an image of the female torso and the boundary of the lips from a facial image. The curves of interest are characterised by ridges or valleys. Key issues in estimation are the ability to navigate across the anatomical surface in three-dimensions, the ability to recognise the relevant boundary and the need to assess the evidence for the presence of the surface feature of interest. The first issue is addressed by the use of principal curves, as an extension of principal components, the second by suitable assessment of curvature and the third by change-point detection. P-spline smoothing is used as an integral part of the methods but adaptations are made to the specific anatomical features of interest. After estimation of the boundary curves, the intermediate surfaces of the anatomical feature of interest can be characterised by surface interpolation. This allows shape variation to be explored using standard methods such as principal components. These tools are applied to a collection of images of women where one breast has been reconstructed after mastectomy and where interest lies in shape differences between the reconstructed and unreconstructed breasts. They are also applied to a collection of lip images where possible differences in shape between males and females are of interest

Virtual Objects on Real Oceans

International audienceAugmented Reality (AR) aims to provide means to integrate virtual objects in a real scene. In that context it is often necessary to recover geometrical information such as objects shapes from the scene in order to add new objects. This paper proposes a semiautomatic method to reconstruct the surface of the ocean from a real ocean scene. A detection algorithm is applied to identify significant waves crestlines. A virtual ocean is then reconstructed using Gerstner model; its parameters are inferred and adjusted by the user to match the crestlines and to provide a smooth reconstruction between adjacent waves. An application is presented to insert a virtual object in the real ocean scene that computes correct occlusions between the ocean surface and the object and uses OpenGL for real-time renderin

An investigation into the dynamical and statistical properties of dominant ocean surface waves using close-range remote sensing

Denne avhandlingen er basert på forskningsresultat som behandler statistiske og dynamiske egenskaper av dominante vinddrevne overflatebølger i åpent hav. Med uttrykket dominante bølger refererer vi her til de største bølgene, med størst energi, i en gitt sjøtilstand. Bølgedrevne prosesser er viktige både i klimasammenheng via atmosfære--hav interaksjon som drives i stor grad av bølgebrytning, samt for kommersiell og rekreasjonell offshorevirksomhet p.g.a. risikoen for å bli utsatt for f.eks. ekstreme enkeltbølger. Både bølgebrytning og ekstrembølgestatistikk er i skrivende stund ufullstendig representert i teoretiske og numeriske modeller. Arbeidet som presenteres i denne avhandlingen undersøker de ovennevnte temaene ved bruk av bølgeobservasjoner som er primært samlet inn på Ekofiskfeltet i den sentrale delen av Nordsjøen. Observasjonsdatasettene består av en langtidstidsserie av laser-altimetermålinger og stereoskopiske videodata fra Ekofisk, samt videomålinger av brytende bølger fra et forskningstokt i nordre Stillehavet. Forskningsresultatene er presentert i artikkelform med to publiserte verk og ett innlevert manuskript. Det blir påvist en tydelig forbindelse mellom økt bølgebrytning og dominante bølgegrupper, et resultat som tidligere har blitt påvist i laboratorie- og modelleksperiment, men sjeldent ved bruk av feltobservasjoner. Tredimensjonale stereo-rekonstruksjoner viser også at ekstreme bølgekammer, både brytende og ikke-brytende, følger nylig utviklet teori om ikke-lineær bølgegruppedynamikk. Dette funnet har konsekvenser f.eks. for estimering av geometriske og kinematiske bølgeegenskaper såsom steilhet og kamhastighet fra endimensjonale tidsseriemålinger. Som følge av en langtidsanalyse av endimensjonal bølgestatistikk blir det vist at enrettet, langkammet og bratt sjø mest sannsynlig leder til ekstreme enkeltbølger med statistiske egenskaper som avviker systematisk fra ordinære statistiske modeller. Tredimensjonal, kortsiktig tid-rom-statistikk av ekstreme bølgekammer blir også undersøkt v.h.a. stereomålingene fra Ekofisk. Her blir det vist at statistiske modeller utvidet fra endimensjonale til tredimensjonale bølgefelt i snitt er velegnet til å beskrive forekomsten av de høyeste bølgekammene, spesielt for relativt store tid-rom segment.The research presented in this thesis characterizes statistical and dynamical aspects of dominant wind-generated surface gravity waves inferred from field observations in intermediate-to-deep water. Dominant waves are the most energetic waves in a sea state, and as such, understanding their behavior is important in both engineering and geophysical contexts. Large waves impart considerable impact forces on marine structures such as oil and gas platforms and offshore wind turbines, and these forces may multiply manyfold when waves break. Wave breaking in deep water, often referred to as whitecapping, is also a key, though incompletely understood, process regulating the transfer of momentum, gas and heat across the air-sea interface, and must thus be accurately parameterized in large-scale weather and climate models. Current theory holds that the wave breaking process is closely linked kinematically and dynamically to the group structure inherent in ocean surface wave fields. Wave group dynamics is also believed to govern the characteristic shape and motion of so-called extreme or rogue waves, whose correct statistical description is central to many offshore activities. The work presented herein shows, using state-of-the-art stereoscopic imaging techniques employed at the Ekofisk platform complex in the central North Sea, that large-scale wave breaking activity in the open ocean is strongly enhanced in dominant wave groups. The topic of wave group-modulated wave breaking has received considerable attention in the past two decades from theoretical, numerical and laboratory perspectives; however, quantitative field studies of the phenomenon remain comparatively rare. The current results also support the general notion that the dominant waves in a given sea state regulate the breaking of shorter waves. The statistics of extreme wave crest elevations is investigated using a novel long-term laser altimeter data set, also located at the Ekofisk field. The validity of the extreme values is verified using a newly developed despiking methodology, and the quality controlled data set, which covers storm events over an 18-year period, is used to investigate the effects of wave steepness and directionality on crest height statistics. Narrow directional spread combined with high wave steepness is found to lead to crest height statistics that deviate the most from standard linear and second-order formulations. Finally, geometric wave shape and crest speed dynamics are analyzed for the highest wave crests encountered in three-dimensional, spatially and temporally resolved segments of the stereo-reconstructed sea surface fields. The directly measured crest steepness is found to conform to the classical breaking limit of Stokes, whereas crest steepness estimated from one-dimensional time series measurements using the linear gravity-wave dispersion relation are systematically higher. This may be at least in part explained by the observation that the directly measured crest speed just before, during and after the moment of maximum crest elevation slows down compared to the linear gravity-wave phase speed estimate. For the first time, the crest speed slowdown is shown with field measurements to apply to both breaking and non-breaking dominant wave crests.Doktorgradsavhandlin

An experimental study of wind-driven surface water transport process pertinent to aircraft icing

Water transport behaviors will significantly influence the icing accretion process during glaze icing conditions. Many important micro-physical processes associated with water transport phenomena, such as film/rivulet formation on the flat and curve surface, surface waves generation, and interaction of runback liquid with local ice roughnesses, are still unclear. In order to elucidate the underlying physics of water transport behaviors under icing conditions, advanced experimental technique capable of providing accurate measurements on the wind-driven thin film/rivulet flows are highly desirable. A novel digital image projection (DIP) system is presented in this work. Using this new technique, a comprehensive experimental study was conducted to quantify the transient behaviors of the wind-driven surface water transport processes pertinent to aircraft icing problems. DIP technique is a further development of digital fringe projection (DFP) technique. In contrast to project sinusoidal patterns, the digital projector projects a grid pattern with known characteristics onto test objects (i.e., water droplet/rivulet flows over icing accreting surfaces). The heights of 3D objects are linear dependent on the grid point displacements between the measurement images of a 3D shape and the reference image of a zero height substrate. Compared with typical DFP measurement system, the DIP technique can significantly reduce the measurement error as well as decrease the requirement of the measurement image quality. After carefully calibrated and validated, the proposed DIP technique was applied to characterize the wind-driven water rivulet flows. Seen from measurement results, the transient motion of rivulet front was found to be significantly influenced by the surface waves\u27 behaviors. The Force Balance (FB) rivulet breaking criteria is further refined and evaluated by the reconstructed tiny rivulet flow structures. Rivulet meandering phenomena and the water mass trapping induced by the meandered water-air contact line were observed. A model based on force balance analysis at the cross-section of meandering rivulet was built to illustrate the meandering instability of wind-driven rivulet flow. In order to examine the effects of the roughness arrays on the surface film flow, i.e., trapped mass effects, which is pertinent to the surface water runback over airfoils/wings with ice roughness, the DIP technique was used to quantify the transient behavior of wind-driven film flow over a surface with roughness arrays. While surface water mass trapping was observed clearly right downstream of the roughness elements, some other interesting features about the water film flow within roughness elements were also revealed clearly from the quantitative DIP measurements, which were found to agree well with those previous numerical studies. The water runback process on an airfoil surface was reconstructed by the DIP technique. The measurement results clearly revealed that, after impinged on the leading edge of the NACA0012 airfoil, the micro-sized water droplets would coalesce to form a thin water film in the region near the leading edge of the airfoil. The formation of rivulets was found to be time-dependent process and relies on the initial water runback flow structure. The film thickness scaling law is evaluated by the time-average measurements of the film thickness. The measurement results show good consistent with the analytical scaling predictions