6 research outputs found

    Image-Based Classification of Double-Barred Beach States Using a Convolutional Neural Network and Transfer Learning

    Get PDF
    Nearshore sandbars characterize many sandy coasts, and unravelling their dynamics is crucial to understanding nearshore sediment pathways. Sandbar morphologies exhibit complex patterns that can be classified into distinct states. The tremendous progress in data-driven learning in image recognition has recently led to the first automated classification of single-barred beach states from Argus imagery using a Convolutional Neural Network (CNN). Herein, we extend this method for the classification of beach states in a double-barred system. We used transfer learning to fine-tune the pre-trained network of ResNet50. Our data consisted of labelled single-bar time-averaged images from the beaches of Narrabeen (Australia) and Duck (US), complemented by 9+ years of daily averaged low-tide images of the double-barred beach of the Gold Coast (Australia). We assessed seven different CNNs, of which each model was tested on the test data from the location where its training data came from, the self-tests, and on the test data of alternate, unseen locations, the transfer-tests. When the model trained on the single-barred data of both Duck and Narrabeen was tested on unseen data of the double-barred Gold Coast, we achieved relatively low performances as measured by F1 scores. In contrast, models trained with only the double-barred beach data showed comparable skill in the self-tests with that of the single-barred models. We incrementally added data with labels from the inner or outer bar of the Gold Coast to the training data from both single-barred beaches, and trained models with both single- and double-barred data. The tests with these models showed that which bar the labels used for training the model mattered. The training with the outer bar labels led to overall higher performances, except at the inner bar. Furthermore, only 10% of additional data with the outer bar labels was needed for reasonable transferability, compared to the 20% of additional data needed with the inner bar labels. Additionally, when trained with data from multiple locations, more data from a new location did not always positively affect the model’s performance on other locations. However, the larger diversity of images coming from more locations allowed the transferability of the model to the locations from where new training data were added

    Labeling poststorm coastal imagery for machine learning: measurement of interrater agreement

    Get PDF
    © The Author(s), 2021. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Goldstein, E. B., Buscombe, D., Lazarus, E. D., Mohanty, S. D., Rafique, S. N., Anarde, K. A., Ashton, A. D., Beuzen, T., Castagno, K. A., Cohn, N., Conlin, M. P., Ellenson, A., Gillen, M., Hovenga, P. A., Over, J.-S. R., Palermo, R., Ratliff, K. M., Reeves, I. R. B., Sanborn, L. H., Straub, J. A., Taylor, L. A., Wallace E. J., Warrick, J., Wernette, P., Williams, H. E. Labeling poststorm coastal imagery for machine learning: measurement of interrater agreement. Earth and Space Science, 8(9), (2021): e2021EA001896, https://doi.org/10.1029/2021EA001896.Classifying images using supervised machine learning (ML) relies on labeled training data—classes or text descriptions, for example, associated with each image. Data-driven models are only as good as the data used for training, and this points to the importance of high-quality labeled data for developing a ML model that has predictive skill. Labeling data is typically a time-consuming, manual process. Here, we investigate the process of labeling data, with a specific focus on coastal aerial imagery captured in the wake of hurricanes that affected the Atlantic and Gulf Coasts of the United States. The imagery data set is a rich observational record of storm impacts and coastal change, but the imagery requires labeling to render that information accessible. We created an online interface that served labelers a stream of images and a fixed set of questions. A total of 1,600 images were labeled by at least two or as many as seven coastal scientists. We used the resulting data set to investigate interrater agreement: the extent to which labelers labeled each image similarly. Interrater agreement scores, assessed with percent agreement and Krippendorff's alpha, are higher when the questions posed to labelers are relatively simple, when the labelers are provided with a user manual, and when images are smaller. Experiments in interrater agreement point toward the benefit of multiple labelers for understanding the uncertainty in labeling data for machine learning research.The authors gratefully acknowledge support from the U.S. Geological Survey (G20AC00403 to EBG and SDM), NSF (1953412 to EBG and SDM; 1939954 to EBG), Microsoft AI for Earth (to EBG and SDM), The Leverhulme Trust (RPG-2018-282 to EDL and EBG), and an Early Career Research Fellowship from the Gulf Research Program of the National Academies of Sciences, Engineering, and Medicine (to EBG). U.S. Geological Survey researchers (DB, J-SRO, JW, and PW) were supported by the U.S. Geological Survey Coastal and Marine Hazards and Resources Program as part of the response and recovery efforts under congressional appropriations through the Additional Supplemental Appropriations for Disaster Relief Act, 2019 (Public Law 116-20; 133 Stat. 871)

    Beach State Recognition Using Argus Imagery and Convolutional Neural Networks

    No full text
    Nearshore morphology is a key driver in wave breaking and the resulting nearshore circulation, recreational safety, and nutrient dispersion. Morphology persists within the nearshore in specific shapes that can be classified into equilibrium states. Equilibrium states convey qualitative information about bathymetry and relevant physical processes. While nearshore bathymetry is a challenge to collect, much information about the underlying bathymetry can be gained from remote sensing of the surfzone. This study presents a new method to automatically classify beach state from Argus daytimexposure imagery using a machine learning technique called convolutional neural networks (CNNs). The CNN processed imagery from two locations: Narrabeen, New South Wales, Australia and Duck, North Carolina, USA. Three different CNN models are examined, one trained at Narrabeen, one at Duck, and one trained at both locations. Each model was tested at the location where it was trained in a self-test, and the single-beach models were tested at the location where it was not trained in a transfer-test. For the self-tests, skill (as measured by the F-score) was comparable to expert agreement (CNN F-values at Duck = 0.80 and Narrabeen = 0.59). For the transfer-tests, the CNN model skill was reduced by 24–48%, suggesting the algorithm requires additional local data to improve transferability performance. Transferability tests showed that comparable F-scores (within 10%) to the self-trained cases can be achieved at both locations when at least 25% of the training data is from each site. This suggests that if applied to additional locations, a CNN model trained at one location may be skillful at new sites with limited new imagery data needed. Finally, a CNN visualization technique (Guided-Grad-CAM) confirmed that the CNN determined classifications using image regions (e.g., incised rip channels, terraces) that were consistent with beach state labelling rules

    Image-Based Classification of Double-Barred Beach States Using a Convolutional Neural Network and Transfer Learning

    No full text
    Nearshore sandbars characterize many sandy coasts, and unravelling their dynamics is crucial to understanding nearshore sediment pathways. Sandbar morphologies exhibit complex patterns that can be classified into distinct states. The tremendous progress in data-driven learning in image recognition has recently led to the first automated classification of single-barred beach states from Argus imagery using a Convolutional Neural Network (CNN). Herein, we extend this method for the classification of beach states in a double-barred system. We used transfer learning to fine-tune the pre-trained network of ResNet50. Our data consisted of labelled single-bar time-averaged images from the beaches of Narrabeen (Australia) and Duck (US), complemented by 9+ years of daily averaged low-tide images of the double-barred beach of the Gold Coast (Australia). We assessed seven different CNNs, of which each model was tested on the test data from the location where its training data came from, the self-tests, and on the test data of alternate, unseen locations, the transfer-tests. When the model trained on the single-barred data of both Duck and Narrabeen was tested on unseen data of the double-barred Gold Coast, we achieved relatively low performances as measured by F1 scores. In contrast, models trained with only the double-barred beach data showed comparable skill in the self-tests with that of the single-barred models. We incrementally added data with labels from the inner or outer bar of the Gold Coast to the training data from both single-barred beaches, and trained models with both single- and double-barred data. The tests with these models showed that which bar the labels used for training the model mattered. The training with the outer bar labels led to overall higher performances, except at the inner bar. Furthermore, only 10% of additional data with the outer bar labels was needed for reasonable transferability, compared to the 20% of additional data needed with the inner bar labels. Additionally, when trained with data from multiple locations, more data from a new location did not always positively affect the model’s performance on other locations. However, the larger diversity of images coming from more locations allowed the transferability of the model to the locations from where new training data were added

    Labels for Emergency Response Imagery from Hurricane Florence, Hurricane Michael, and Hurricane Isaias

    No full text
    The csv files contain&nbsp;human-generated labels for Emergency Response Imagery collected by US National Oceanic and Atmospheric Administration (NOAA) after Hurricane Florence (2018), Hurricane Michael (2018) and Hurricane Isaias (2020). All authors contributed to labeling the imagery. All labeling was done with an open-source labeling tool (Rafique et al., 2020). All csv files provide&nbsp;the userID (the ID of the anonymous labeler), the NOAA flight, the NOAA image, and 6 labels &mdash; allWater (if the image was all water), devType (if the image had buildings/development), washoverType (if the image had washover deposits), dmgType (if the image showed damage to built environment), impactType (if the labeler could identify the coastal impact, using the Storm Impact Scale from Sallenger, 2000), and terrainType (the type of physical environment). Images labeled here correspond to 3 NOAA flights &mdash; Florence 20180917a , Michael 20181011a, Isaias 20200804a. These images can be downloaded directly from NOAA (https://storms.ngs.noaa.gov/) or using Moretz et al. (2020a, 2020b). There are three csv files: ReleaseData_v3.csv has 6200 labels&nbsp;for 1500 images. These labels were generated by trained coastal scientists. ReleaseDataQuads.csv has 400 labels for 100 images. These labels were generated by trained coastal scientists. The images labeled in this set correspond to original NOAA images that have been split into quadrants. Splitting images was done with ImageMagick. The command used to split the images was:`magick mogrify -crop 2x2@ +repage -path ../quadrants *.jpg` The naming convention corresponds to the image quarter &mdash; the *-0.jpg is upper left, *-1.jpg is upper right, *-2.jpg is lower left, and *-3.jpg is the lower right. ReleaseDataNCE.csv has 400 labels for 100 images. These images were labeled by non-coastal scientists. Note that the 100 images were also labeled by coastal scientists &mdash; those labels can be found in ReleaseData_v3.csv. There is another companion dataset to this, with slightly different labels (Goldstein et al., 2020) </span
    corecore