Automated classification of three-dimensional reconstructions of coral reefs using convolutional neural networks

© The Author(s), 2020. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Hopkinson, B. M., King, A. C., Owen, D. P., Johnson-Roberson, M., Long, M. H., & Bhandarkar, S. M. Automated classification of three-dimensional reconstructions of coral reefs using convolutional neural networks. PLoS One, 15(3), (2020): e0230671, doi: 10.1371/journal.pone.0230671.Coral reefs are biologically diverse and structurally complex ecosystems, which have been severally affected by human actions. Consequently, there is a need for rapid ecological assessment of coral reefs, but current approaches require time consuming manual analysis, either during a dive survey or on images collected during a survey. Reef structural complexity is essential for ecological function but is challenging to measure and often relegated to simple metrics such as rugosity. Recent advances in computer vision and machine learning offer the potential to alleviate some of these limitations. We developed an approach to automatically classify 3D reconstructions of reef sections and assessed the accuracy of this approach. 3D reconstructions of reef sections were generated using commercial Structure-from-Motion software with images extracted from video surveys. To generate a 3D classified map, locations on the 3D reconstruction were mapped back into the original images to extract multiple views of the location. Several approaches were tested to merge information from multiple views of a point into a single classification, all of which used convolutional neural networks to classify or extract features from the images, but differ in the strategy employed for merging information. Approaches to merging information entailed voting, probability averaging, and a learned neural-network layer. All approaches performed similarly achieving overall classification accuracies of ~96% and >90% accuracy on most classes. With this high classification accuracy, these approaches are suitable for many ecological applications.This study was funded by grants from the Alfred P. Sloan Foundation (BMH, BR2014-049; https://sloan.org), and the National Science Foundation (MHL, OCE-1657727; https://www.nsf.gov). The funders had no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript

Spatial image modulation to improve performance of computed tomography imaging spectrometer

Computed tomography imaging spectrometers ("CTIS"s) having patterns for imposing spatial structure are provided. The pattern may be imposed either directly on the object scene being imaged or at the field stop aperture. The use of the pattern improves the accuracy of the captured spatial and spectral information

Color camera computed tomography imaging spectrometer for improved spatial-spectral image accuracy

Computed tomography imaging spectrometers ("CTIS"s) having color focal plane array detectors are provided. The color FPA detector may comprise a digital color camera including a digital image sensor, such as a Foveon X3.RTM. digital image sensor or a Bayer color filter mosaic. In another embodiment, the CTIS includes a pattern imposed either directly on the object scene being imaged or at the field stop aperture. The use of a color FPA detector and the pattern improves the accuracy of the captured spatial and spectral information

Single-lens computed tomography imaging spectrometer and method of capturing spatial and spectral information

Computed tomography imaging spectrometers ("CTISs") employing a single lens are provided. The CTISs may be either transmissive or reflective, and the single lens is either configured to transmit and receive uncollimated light (in transmissive systems), or is configured to reflect and receive uncollimated light (in reflective systems). An exemplary transmissive CTIS includes a focal plane array detector, a single lens configured to transmit and receive uncollimated light, a two-dimensional grating, and a field stop aperture. An exemplary reflective CTIS includes a focal plane array detector, a single mirror configured to reflect and receive uncollimated light, a two-dimensional grating, and a field stop aperture

Spatial Modulation Improves Performance in CTIS

Suitably formulated spatial modulation of a scene imaged by a computed-tomography imaging spectrometer (CTIS) has been found to be useful as a means of improving the imaging performance of the CTIS. As used here, "spatial modulation" signifies the imposition of additional, artificial structure on a scene from within the CTIS optics. The basic principles of a CTIS were described in "Improvements in Computed- Tomography Imaging Spectrometry" (NPO-20561) NASA Tech Briefs, Vol. 24, No. 12 (December 2000), page 38 and "All-Reflective Computed-Tomography Imaging Spectrometers" (NPO-20836), NASA Tech Briefs, Vol. 26, No. 11 (November 2002), page 7a. To recapitulate: A CTIS offers capabilities for imaging a scene with spatial, spectral, and temporal resolution. The spectral disperser in a CTIS is a two-dimensional diffraction grating. It is positioned between two relay lenses (or on one of two relay mirrors) in a video imaging system. If the disperser were removed, the system would produce ordinary images of the scene in its field of view. In the presence of the grating, the image on the focal plane of the system contains both spectral and spatial information because the multiple diffraction orders of the grating give rise to multiple, spectrally dispersed images of the scene. By use of algorithms adapted from computed tomography, the image on the focal plane can be processed into an image cube a three-dimensional collection of data on the image intensity as a function of the two spatial dimensions (x and y) in the scene and of wavelength (lambda). Thus, both spectrally and spatially resolved information on the scene at a given instant of time can be obtained, without scanning, from a single snapshot; this is what makes the CTIS such a potentially powerful tool for spatially, spectrally, and temporally resolved imaging. A CTIS performs poorly in imaging some types of scenes in particular, scenes that contain little spatial or spectral variation. The computed spectra of such scenes tend to approximate correct values to within acceptably small errors near the edges of the field of view but to be poor approximations away from the edges. The additional structure imposed on a scene according to the present method enables the CTIS algorithms to reconstruct acceptable approximations of the spectral data throughout the scene

The Red-Cockaded Woodpecker Cavity Tree: A Very Special Pine

The adaptation of red-cockaded woodpeckers (Picoides borealis) to fire-maintained southern pine ecosystems has included the development of behaviors that permit the species to use living pines for their cavity trees. Their adaptation to pine ecosystems has also involved a major adjustment in the species\u27 breeding system to cooperative breeding, probably in response to the extended time period required to excavate a completed cavity in a living pine and the relative rarity of completed cavities for nesting. The characteristics of live pines make them variable in their suitability as cavity trees, leading to the evolution of selection behavior among woodpeckers. Red-cockaded woodpeckers require a very special type of pine for their cavity tree. Potential cavity trees must be sufficiently old because only older pines have heartwood of sufficient diameter to physically house a woodpecker cavity without breaching the resin producing sapwood. Older pines also have a larger diameter of heartwood higher in the pine, permitting higher cavity placement, well away from frequent fires. Older pines also have a higher occurrence rate of red heart fungus (Phellinus pini), which decays the heartwood allowing cavity excavation to proceed more quickly. The potential cavity tree also needs to have relatively thin sapwood, which reduces the time the woodpecker must spend excavating through living xylem tissue that exudes sticky pine resin when pecked. Red-cockaded woodpeckers scale loose bark from the bole of their cavity trees and excavate resin wells above and below cavity entrances. These behaviors create a resin barrier that is very effective in deterring predation by rat snakes (Elaphe spp.). Thus, the ability of pines to produce adequate resin is also important to the woodpecker. Red-cockaded wood- peckers can detect the pine\u27s ability to produce resin and select pines that are high producers. Higher yields of resin likely create better barriers against rat snakes. The socially dominant breeding male red-cockaded woodpecker selects the cavity tree that produces the most resin for its roost tree, which during spring becomes the group\u27s nest tree. Our recent research suggests that red-cockaded woodpeckers also select pines with particular resin chemistries. High concentrations of diterpenes may increase resin viscosity, stickiness, irritability, or other factors that may be important for creating a barrier against rat snakes

Accelerated Hatching of Southern Leopard Frog (Rana sphenocephala) Eggs in Response to the Presence of a Crayfish (Procambarus nigrocinctus) Predator

Phenotypic plasticity, such as morphological and behavioral changes in response to predators, is common in larval anurans. Less is known about inducible defenses in the embryonic stages of development. We investigated the predation risk imposed by crayfish (Procambarus nigrocinctus) on southern leopard frog (Rana sphenocephala) eggs aud whether crayfish presence induces a change in the timing of hatching of R. sphenocephala eggs. We found that crayfish significantly reduce the hatching success of R. sphenocephala eggs by eating them and that eggs hatch significantly faster in the presence of crayfish than when crayfish are not present. We also found that the nonlethal presence of crayfish (caged with no access to eggs) induced accelerated hatching, indicating that injured conspecifics are not required to elicit the response. Reception of chemical cues produced or released by crayfish may play an important role in survival of R. sphenocephala eggs