Deep Learning-Based Object Detection in Maritime Unmanned Aerial Vehicle Imagery: Review and Experimental Comparisons

With the advancement of maritime unmanned aerial vehicles (UAVs) and deep learning technologies, the application of UAV-based object detection has become increasingly significant in the fields of maritime industry and ocean engineering. Endowed with intelligent sensing capabilities, the maritime UAVs enable effective and efficient maritime surveillance. To further promote the development of maritime UAV-based object detection, this paper provides a comprehensive review of challenges, relative methods, and UAV aerial datasets. Specifically, in this work, we first briefly summarize four challenges for object detection on maritime UAVs, i.e., object feature diversity, device limitation, maritime environment variability, and dataset scarcity. We then focus on computational methods to improve maritime UAV-based object detection performance in terms of scale-aware, small object detection, view-aware, rotated object detection, lightweight methods, and others. Next, we review the UAV aerial image/video datasets and propose a maritime UAV aerial dataset named MS2ship for ship detection. Furthermore, we conduct a series of experiments to present the performance evaluation and robustness analysis of object detection methods on maritime datasets. Eventually, we give the discussion and outlook on future works for maritime UAV-based object detection. The MS2ship dataset is available at \href{https://github.com/zcj234/MS2ship}{https://github.com/zcj234/MS2ship}.Comment: 32 pages, 18 figure

Introduction to the Special Issue on Sustainable Solutions for the Intelligent Transportation Systems

The intelligent transportation systems improve the transportation system’s operational efficiency and enhance its safety and reliability by high-tech means such as information technology, control technology, and computer technology. In recent years, sustainable development has become an important topic in intelligent transportation’s development, including new infrastructure and energy distribution, new energy vehicles and new transportation systems, and the development of low-carbon and intelligent transportation equipment. New energy vehicles’ development is a significant part of green transportation, and its automation performance improvement is vital for smart transportation. The development of intelligent transportation and green, low-carbon, and intelligent transportation equipment needs to be promoted, a significant feature of transportation development in the future. For intelligent infrastructure and energy distribution facilities, the electricity for popular electric vehicles and renewable energy, such as nuclear power and hydrogen power, should be considered

ORCA-SPY enables killer whale sound source simulation, detection, classification and localization using an integrated deep learning-based segmentation

Acoustic identification of vocalizing individuals opens up new and deeper insights into animal communications, such as individual-/group-specific dialects, turn-taking events, and dialogs. However, establishing an association between an individual animal and its emitted signal is usually non-trivial, especially for animals underwater. Consequently, a collection of marine species-, array-, and position-specific ground truth localization data is extremely challenging, which strongly limits possibilities to evaluate localization methods beforehand or at all. This study presents ORCA-SPY, a fully-automated sound source simulation, classification and localization framework for passive killer whale (Orcinus orca) acoustic monitoring that is embedded into PAMGuard, a widely used bioacoustic software toolkit. ORCA-SPY enables array- and position-specific multichannel audio stream generation to simulate real-world ground truth killer whale localization data and provides a hybrid sound source identification approach integrating ANIMAL-SPOT, a state-of-the-art deep learning-based orca detection network, followed by downstream Time-Difference-Of-Arrival localization. ORCA-SPY was evaluated on simulated multichannel underwater audio streams including various killer whale vocalization events within a large-scale experimental setup benefiting from previous real-world fieldwork experience. Across all 58,320 embedded vocalizing killer whale events, subject to various hydrophone array geometries, call types, distances, and noise conditions responsible for a signal-to-noise ratio varying from −14.2 dB to 3 dB, a detection rate of 94.0 % was achieved with an average localization error of 7.01∘. ORCA-SPY was field-tested on Lake Stechlin in Brandenburg Germany under laboratory conditions with a focus on localization. During the field test, 3889 localization events were observed with an average error of 29.19∘ and a median error of 17.54∘. ORCA-SPY was deployed successfully during the DeepAL fieldwork 2022 expedition (DLFW22) in Northern British Columbia, with a mean average error of 20.01∘ and a median error of 11.01∘ across 503 localization events. ORCA-SPY is an open-source and publicly available software framework, which can be adapted to various recording conditions as well as animal species

Advanced study of coastal zone oceanographic requirements for ERTS E and F

Earth Resources Technology Satellites E and F orbits and remote sensor instruments for coastal oceanographic data collectio

Ocean carbon from space: Current status and priorities for the next decade

The ocean plays a central role in modulating the Earth’s carbon cycle. Monitoring how the ocean carbon cycle is changing is fundamental to managing climate change. Satellite remote sensing is currently our best tool for viewing the ocean surface globally and systematically, at high spatial and temporal resolutions, and the past few decades have seen an exponential growth in studies utilising satellite data for ocean carbon research. Satellite-based observations must be combined with in-situ observations and models, to obtain a comprehensive view of ocean carbon pools and fluxes. To help prioritise future research in this area, a workshop was organised that assembled leading experts working on the topic, from around the world, including remote-sensing scientists, field scientists and modellers, with the goal to articulate a collective view of the current status of ocean carbon research, identify gaps in knowledge, and formulate a scientific roadmap for the next decade, with an emphasis on evaluating where satellite remote sensing may contribute. A total of 449 scientists and stakeholders participated (with balanced gender representation), from North and South America, Europe, Asia, Africa, and Oceania. Sessions targeted both inorganic and organic pools of carbon in the ocean, in both dissolved and particulate form, as well as major fluxes of carbon between reservoirs (e.g., primary production) and at interfaces (e.g., air-sea and land–ocean). Extreme events, blue carbon and carbon budgeting were also key topics discussed. Emerging priorities identified include: expanding the networks and quality of in-situ observations; improved satellite retrievals; improved uncertainty quantification; improved understanding of vertical distributions; integration with models; improved techniques to bridge spatial and temporal scales of the different data sources; and improved fundamental understanding of the ocean carbon cycle, and of the interactions among pools of carbon and light. We also report on priorities for the specific pools and fluxes studied, and highlight issues and concerns that arose during discussions, such as the need to consider the environmental impact of satellites or space activities; the role satellites can play in monitoring ocean carbon dioxide removal approaches; economic valuation of the satellite based information; to consider how satellites can contribute to monitoring cycles of other important climatically-relevant compounds and elements; to promote diversity and inclusivity in ocean carbon research; to bring together communities working on different aspects of planetary carbon; maximising use of international bodies; to follow an open science approach; to explore new and innovative ways to remotely monitor ocean carbon; and to harness quantum computing. Overall, this paper provides a comprehensive scientific roadmap for the next decade on how satellite remote sensing could help monitor the ocean carbon cycle, and its links to the other domains, such as terrestrial and atmosphere

Ocean carbon from space: Current status and priorities for the next decade

The ocean plays a central role in modulating the Earth\u27s carbon cycle. Monitoring how the ocean carbon cycle is changing is fundamental to managing climate change. Satellite remote sensing is currently our best tool for viewing the ocean surface globally and systematically, at high spatial and temporal resolutions, and the past few decades have seen an exponential growth in studies utilising satellite data for ocean carbon research. Satellite-based observations must be combined with in-situ observations and models, to obtain a comprehensive view of ocean carbon pools and fluxes. To help prioritise future research in this area, a workshop was organised that assembled leading experts working on the topic, from around the world, including remote-sensing scientists, field scientists and modellers, with the goal to articulate a collective view of the current status of ocean carbon research, identify gaps in knowledge, and formulate a scientific roadmap for the next decade, with an emphasis on evaluating where satellite remote sensing may contribute. A total of 449 scientists and stakeholders participated (with balanced gender representation), from North and South America, Europe, Asia, Africa, and Oceania. Sessions targeted both inorganic and organic pools of carbon in the ocean, in both dissolved and particulate form, as well as major fluxes of carbon between reservoirs (e.g., primary production) and at interfaces (e.g., air-sea and land–ocean). Extreme events, blue carbon and carbon budgeting were also key topics discussed. Emerging priorities identified include: expanding the networks and quality of in-situ observations; improved satellite retrievals; improved uncertainty quantification; improved understanding of vertical distributions; integration with models; improved techniques to bridge spatial and temporal scales of the different data sources; and improved fundamental understanding of the ocean carbon cycle, and of the interactions among pools of carbon and light. We also report on priorities for the specific pools and fluxes studied, and highlight issues and concerns that arose during discussions, such as the need to consider the environmental impact of satellites or space activities; the role satellites can play in monitoring ocean carbon dioxide removal approaches; economic valuation of the satellite based information; to consider how satellites can contribute to monitoring cycles of other important climatically-relevant compounds and elements; to promote diversity and inclusivity in ocean carbon research; to bring together communities working on different aspects of planetary carbon; maximising use of international bodies; to follow an open science approach; to explore new and innovative ways to remotely monitor ocean carbon; and to harness quantum computing. Overall, this paper provides a comprehensive scientific roadmap for the next decade on how satellite remote sensing could help monitor the ocean carbon cycle, and its links to the other domains, such as terrestrial and atmosphere

Optimizing a dynamic fossil fuel CO2 emission model with CTDAS (CarbonTracker Data Assimilation Shell, v1.0) for an urban area using atmospheric observations of CO2, CO, NOx, and SO2

We present a modelling framework for fossil fuel CO2 emissions in an urban environment, which allows constraints from emission inventories to be combined with atmospheric observations of CO2 and its co-emitted species CO, NOx , and SO2. Rather than a static assignment of average emission rates to each unit area of the urban domain, the fossil fuel emissions we use are dynamic: they vary in time and space in relation to data that describe or approximate the activity within a sector, such as traffic density, power demand, 2m temperature (as proxy for heating demand), and sunlight and wind speed (as proxies for renewable energy supply). Through inverse modelling, we optimize the relationships between these activity data and the resulting emissions of all species within the dynamic fossil fuel emission model, based on atmospheric mole fraction observations. The advantage of this novel approach is that the optimized parameters (emission factors and emission ratios, N D 44) in this dynamic emission model (a) vary much less over space and time, (b) allow for a physical interpretation of mean and uncertainty, and (c) have better defined uncertainties and covariance structure. This makes them more suited to extrapolate, optimize, and interpret than the gridded emissions themselves. The merits of this approach are investigated using a pseudo-observation-based ensemble Kalman filter inversion set-up for the Dutch Rijnmond area at 1km-1km resolution. We find that the fossil fuel emission model approximates the gridded emissions well (annual mean differences < 2 %, hourly temporal r2 D 0:21-0.95), while reported errors in the underlying parameters allow a full covariance structure to be created readily. Propagating this error structure into atmospheric mole fractions shows a strong dominance of a few large sectors and a few dominant uncertainties, most notably the emission ratios of the various gases considered. If the prior emission ratios are either sufficiently well-known or well constrained from a dense observation network, we find that including observations of co-emitted species improves our ability to estimate emissions per sector relative to using CO2 mole fractions only. Nevertheless, the total CO2 emissions can be well constrained with CO2 as the only tracer in the inversion. Because some sectors are sampled only sparsely over a day, we find that propagating solutions from day-to-day leads to largest uncertainty reduction and smallest CO2 residuals over the 14 consecutive days considered. Although we can technically estimate the temporal distribution of some emission categories like shipping separate from their total magnitude, the controlling parameters are difficult to distinguish. Overall, we conclude that our new system looks promising for application in verification studies, provided that reliable urban atmospheric transport fields and reasonable a priori emission ratios for CO2 and its co-emitted species can be produced

ROTracker: a novel MMW radar-based object tracking method for unmanned surface vehicle in offshore environments

Unmanned surface vehicles (USVs) offer significant value through their capability to undertake hazardous and time-consuming missions across water surfaces. Recently, as the application of USVs has been extended to nearshore waterways, object tracking is vital to the safe navigation of USVs in offshore scenes. However, existing tracking systems for USVs are mainly based on cameras or LiDAR sensors, which suffer from drawbacks such as lack of depth perception or high deployment costs. In contrast, millimeter-wave (MMW) radar offers advantages in terms of low cost and robustness in all weather and lighting conditions. In this work, to construct a robust and low-cost tracking system for USVs in complex offshore scenes, we propose a novel MMW radar-based object tracking method (ROTracker). The proposed ROTracker combines the physical properties of MMW radar with traditional tracking systems. Specifically, we introduce the radar Doppler velocity and a designed motion discriminator to improve the robustness of the tracking system toward low-speed targets. Moreover, we conducted real-world experiments to validate the efficacy of the proposed ROTracker. Compared to other baseline methods, ROTracker achieves excellent multiple object tracking accuracy in terms of 91.9% in our collected dataset. The experimental results demonstrated that the proposed ROTracker has significant application potential in both accuracy and efficiency for USVs, addressing the challenges posed by complex nearshore environments