Validation of the F-POD—a fully automated cetacean monitoring system

The F-POD, an echolocation-click logging device, is commonly used for passive acoustic monitoring of cetaceans. This paper presents the first assessment of the error-rate of fully automated analysis by this system, a description of the F-POD hardware, and a description of the KERNO-F v1.0 classifier which identifies click trains. Since 2020, twenty F-POD loggers have been used in the BlackCeTrends project by research teams from Bulgaria, Georgia, Romania, Türkiye, and Ukraine with the aim of investigating trends of relative abundance in populations of cetaceans of the Black Sea. Acoustic data from this project analysed here comprises 9 billion raw data clicks in total, of which 297 million were classified by KERNO-F as Narrow Band High Frequency (NBHF) clicks (harbour porpoise clicks) and 91 million as dolphin clicks. Such data volumes require a reliable automated system of analysis, which we describe. A total of 16,805 Detection Positive Minutes (DPM) were individually inspected and assessed by a visual check of click train characteristics in each DPM. To assess the overall error rate in each species group we investigated 2,000 DPM classified as having NBHF clicks and 2,000 DPM classified as having dolphin clicks. The fraction of NBHF DPM containing misclassified NBHF trains was less than 0.1% and for dolphins the corresponding error-rate was 0.97%. For both species groups (harbour porpoises and dolphins), these error-rates are acceptable for further study of cetaceans in the Black Sea using the automated classification without further editing of the data. The main sources of errors were 0.17% of boat sonar DPMs misclassified as harbour porpoises, and 0.14% of harbour porpoise DPMs misclassified as dolphins. The potential to estimate the rate at which these sources generate errors makes possible a new predictive approach to overall error estimation.Publisher PDFPeer reviewe

S1 File -

The F-POD, an echolocation-click logging device, is commonly used for passive acoustic monitoring of cetaceans. This paper presents the first assessment of the error-rate of fully automated analysis by this system, a description of the F-POD hardware, and a description of the KERNO-F v1.0 classifier which identifies click trains. Since 2020, twenty F-POD loggers have been used in the BlackCeTrends project by research teams from Bulgaria, Georgia, Romania, Türkiye, and Ukraine with the aim of investigating trends of relative abundance in populations of cetaceans of the Black Sea. Acoustic data from this project analysed here comprises 9 billion raw data clicks in total, of which 297 million were classified by KERNO-F as Narrow Band High Frequency (NBHF) clicks (harbour porpoise clicks) and 91 million as dolphin clicks. Such data volumes require a reliable automated system of analysis, which we describe. A total of 16,805 Detection Positive Minutes (DPM) were individually inspected and assessed by a visual check of click train characteristics in each DPM. To assess the overall error rate in each species group we investigated 2,000 DPM classified as having NBHF clicks and 2,000 DPM classified as having dolphin clicks. The fraction of NBHF DPM containing misclassified NBHF trains was less than 0.1% and for dolphins the corresponding error-rate was 0.97%. For both species groups (harbour porpoises and dolphins), these error-rates are acceptable for further study of cetaceans in the Black Sea using the automated classification without further editing of the data. The main sources of errors were 0.17% of boat sonar DPMs misclassified as harbour porpoises, and 0.14% of harbour porpoise DPMs misclassified as dolphins. The potential to estimate the rate at which these sources generate errors makes possible a new predictive approach to overall error estimation.</div

Time sequence of a porpoise encounter shown in a raw data file, starting top left.

The horizontal axis shows all clicks in 15 minutes of data in three panels of 5 minutes duration, marked by vertical dashed lines at each minute. The vertical axis is the narrow-band high frequency index (NBHF index–this represents how closely each click matches a typical NBHF clicks as produced by a porpoise). Each coloured vertical line is a click but many overlay others at this time resolution. The first section, marked in green, shows background noise. During the next 7minute section, marked in red, a series of increasingly distinct groups of clicks with higher values of NBHF index are logged and then the pattern reverts rapidly to the background noise pattern, again marked in green. This evidence of a typical encounter adds support to the classification of a train within that likely encounter as being a porpoise click train.</p

Map of the locations of F-PODs in the Black Sea.

Information about locations of F-PODs, numbers, depths, and seabed type is given in Table 1 Locations in the S2 File.</p

Distribution of Dolphin error rates across files.

The F-POD, an echolocation-click logging device, is commonly used for passive acoustic monitoring of cetaceans. This paper presents the first assessment of the error-rate of fully automated analysis by this system, a description of the F-POD hardware, and a description of the KERNO-F v1.0 classifier which identifies click trains. Since 2020, twenty F-POD loggers have been used in the BlackCeTrends project by research teams from Bulgaria, Georgia, Romania, Türkiye, and Ukraine with the aim of investigating trends of relative abundance in populations of cetaceans of the Black Sea. Acoustic data from this project analysed here comprises 9 billion raw data clicks in total, of which 297 million were classified by KERNO-F as Narrow Band High Frequency (NBHF) clicks (harbour porpoise clicks) and 91 million as dolphin clicks. Such data volumes require a reliable automated system of analysis, which we describe. A total of 16,805 Detection Positive Minutes (DPM) were individually inspected and assessed by a visual check of click train characteristics in each DPM. To assess the overall error rate in each species group we investigated 2,000 DPM classified as having NBHF clicks and 2,000 DPM classified as having dolphin clicks. The fraction of NBHF DPM containing misclassified NBHF trains was less than 0.1% and for dolphins the corresponding error-rate was 0.97%. For both species groups (harbour porpoises and dolphins), these error-rates are acceptable for further study of cetaceans in the Black Sea using the automated classification without further editing of the data. The main sources of errors were 0.17% of boat sonar DPMs misclassified as harbour porpoises, and 0.14% of harbour porpoise DPMs misclassified as dolphins. The potential to estimate the rate at which these sources generate errors makes possible a new predictive approach to overall error estimation.</div

Harbour porpoise smooth amplitude profile.

Upper panel: Amplitude of clicks in a porpoise click train showing the characteristic smooth amplitude profiles. Lower panel: clicks without a smooth amplitude profile which is typical of clicks from unrelated sources.</p

Sightings of Cetaceans in the waters of Yuzhny Sea Port (Hryhorivsky Estuary, Black Sea) in 2015–2016

The use of transformed coastal ecosystems by the Black Sea cetaceans is an important, but little studied issue. In 2015–2016, coastal visual observations of cetaceans were conducted in the waters of Yuzhny Sea Port. The area was regularly used by common dolphins (they were sighted in 24 days of 64) and also by harbour porpoises (7 sightings). Group sizes of the common dolphins reached up to 25 individuals (Median = 7), and of the harbour porpoises — up to 5 (Median = 4); calves and juveniles were present in most groups. The primary behaviour of cetaceans was feeding. Because of significant concentrations of fish in Hryhorivsky Estuary, the area of Yuzhny Sea Port is an important habitat for the Black Sea cetaceans

BlackCeTrends project Dataset supporting the paper "Validation of the F-POD – a fully automated cetacean monitoring system".

BlackCeTrends - Large coverage initiative for acoustic survey of cetaceans in the Black Sea An international project “BlackCeTrends” has been launched, in which six institutions from all around the Black Sea: Mare Nostrum NGO (Romania), Green Balkans NGO (Bulgaria), UkrSCES &amp; BioEcoLinks (Ukraine), Turkish Marine Research Foundation TUDAV (Turkey) and Ilia State University (Georgia), together with British acoustic device manufacturer Chelonia Ltd. with the objective of assessing the trends in Black Sea cetaceans using a passive acoustic monitoring method. The main activity of the project is collection of acoustic data to understand the presence of cetaceans, especially harbour porpoises, by deploying fully automated passive acoustic monitoring instruments of the new generation, F-PODs. F-PODs are provided for free for a permanent loan as long as they are in use by Chelonia Ltd. They are designed specifically to detect and record the high-frequency sounds of dolphins and porpoises, and they were newly developed from C-PODs, which have been extensively used in the North and Baltic Seas. The collected data are downloaded periodically, approximately every 1-4 months, and the results are processed by specialized software. Four or five devices have been provided to each of the partners, of which nine are already in place, in the western part of the Black Sea (Figure 1) and all the others will be deployed in the near future