Challenges in video based object detection in maritime scenario using computer vision

This paper discusses the technical challenges in maritime image processing and machine vision problems for video streams generated by cameras. Even well documented problems of horizon detection and registration of frames in a video are very challenging in maritime scenarios. More advanced problems of background subtraction and object detection in video streams are very challenging. Challenges arising from the dynamic nature of the background, unavailability of static cues, presence of small objects at distant backgrounds, illumination effects, all contribute to the challenges as discussed here

MARITIME DOMAIN AWARENESS THROUGH THE CHARACTERIZATION OF SHIP BEHAVIOR WITH AIS DATA

Maritime Domain Awareness (MDA), as defined in the 2005 National Strategy for Maritime Security, is the “effective understanding of anything associated with the global maritime domain that could impact the security, safety, economy, or environment of the United States.” Thus, it is imperative for the U.S. Navy to develop approaches that enhance understanding of the maritime domain in order to maintain operational effectiveness. One such way to enhance this understanding is to develop approaches that automate the analysis of Automatic Identification System (AIS) data to characterize the behavior of ships in the maritime domain. By the sheer amount of AIS data available, it quickly becomes challenging for a human operator to identify ship behaviors throughout the world. When timeliness is important for decision makers, it becomes even more important that the characterization of ship behavior is done quickly and accurately to identify potential issues or threats. Thus, a major contribution of this thesis is the development of an autonomous machine learning system that characterizes ship behavior quickly and accurately in order to achieve MDA in a particular environment. This includes an autonomous system for the identification of ship tracks in a region. Two major contributions of this work are the development of a taxonomy of ship behaviors, which is currently lacking in the literature, and a report on the characterization of such behaviors through machine learning methods.Ensign, United States NavyApproved for public release. Distribution is unlimited

Sea-Surface Object Detection Based on Electro-Optical Sensors: A Review

Sea-surface object detection is critical for navigation safety of autonomous ships. Electrooptical (EO) sensors, such as video cameras, complement radar on board in detecting small obstacle sea-surface objects. Traditionally, researchers have used horizon detection, background subtraction, and foreground segmentation techniques to detect sea-surface objects. Recently, deep learning-based object detection technologies have been gradually applied to sea-surface object detection. This article demonstrates a comprehensive overview of sea-surface object-detection approaches where the advantages and drawbacks of each technique are compared, covering four essential aspects: EO sensors and image types, traditional object-detection methods, deep learning methods, and maritime datasets collection. In particular, sea-surface object detections based on deep learning methods are thoroughly analyzed and compared with highly influential public datasets introduced as benchmarks to verify the effectiveness of these approaches. The arti

Radar Technology

In this book “Radar Technology”, the chapters are divided into four main topic areas: Topic area 1: “Radar Systems” consists of chapters which treat whole radar systems, environment and target functional chain. Topic area 2: “Radar Applications” shows various applications of radar systems, including meteorological radars, ground penetrating radars and glaciology. Topic area 3: “Radar Functional Chain and Signal Processing” describes several aspects of the radar signal processing. From parameter extraction, target detection over tracking and classification technologies. Topic area 4: “Radar Subsystems and Components” consists of design technology of radar subsystem components like antenna design or waveform design

Ship recognition on the sea surface using aerial images taken by Uav : a deep learning approach

Dissertation submitted in partial fulfilment of the requirements for the Degree of Master of Science in Geospatial TechnologiesOceans are very important for mankind, because they are a very important source of food, they have a very large impact on the global environmental equilibrium, and it is over the oceans that most of the world commerce is done. Thus, maritime surveillance and monitoring, in particular identifying the ships used, is of great importance to oversee activities like fishing, marine transportation, navigation in general, illegal border encroachment, and search and rescue operations. In this thesis, we used images obtained with Unmanned Aerial Vehicles (UAVs) over the Atlantic Ocean to identify what type of ship (if any) is present in a given location. Images generated from UAV cameras suffer from camera motion, scale variability, variability in the sea surface and sun glares. Extracting information from these images is challenging and is mostly done by human operators, but advances in computer vision technology and development of deep learning techniques in recent years have made it possible to do so automatically. We used four of the state-of-art pretrained deep learning network models, namely VGG16, Xception, ResNet and InceptionResNet trained on ImageNet dataset, modified their original structure using transfer learning based fine tuning techniques and then trained them on our dataset to create new models. We managed to achieve very high accuracy (99.6 to 99.9% correct classifications) when classifying the ships that appear on the images of our dataset. With such a high success rate (albeit at the cost of high computing power), we can proceed to implement these algorithms on maritime patrol UAVs, and thus improve Maritime Situational Awareness

Proceedings of the 2011 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

This book is a collection of 15 reviewed technical reports summarizing the presentations at the 2011 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory. The covered topics include image processing, optical signal processing, visual inspection, pattern recognition and classification, human-machine interaction, world and situation modeling, autonomous system localization and mapping, information fusion, and trust propagation in sensor networks

Moving Target Analysis in ISAR Image Sequences with a Multiframe Marked Point Process Model

In this paper we propose a Multiframe Marked Point Process model of line segments and point groups for automatic target structure extraction and tracking in Inverse Synthetic Aperture Radar (ISAR) image sequences. For the purpose of dealing with scatterer scintillations and high speckle noise in the ISAR frames, we obtain the resulting target sequence by an iterative optimization process, which simultaneously considers the observed image data and various prior geometric interaction constraints between the target appearances in the consecutive frames. A detailed quantitative evaluation is performed on 8 real ISAR image sequences of different carrier ship and airplane targets, using a test database containing 545 manually annotated frames

GNSS-based passive radar techniques for maritime surveillance

The improvement of maritime traffic safety and security is a subject of growing interest, since the traffic is constantly increasing. In fact, a large number of human activities take place in maritime domain, varying from cruise and trading ships up to vessels involved in nefarious activities such as piracy, human smuggling or terrorist actions. The systems based on Automatic Identification System (AIS) transponder cannot cope with non-cooperative or non-equipped vessels that instead can be detected, tracked and identified by means of radar system. In particular, passive bistatic radar (PBR) systems can perform these tasks without a dedicated transmitter, since they exploit illuminators of opportunity as transmitters. The lack of a dedicated transmitter makes such systems low cost and suitable to be employed in areas where active sensors cannot be placed such as, for example, marine protected areas. Innovative solutions based on terrestrial transmitters have been considered in order to increase maritime safety and security, but these kinds of sources cannot guarantee a global coverage, such as in open sea. To overcome this problem, the exploitation of global navigation satellites system (GNSS) as transmitters of opportunity is a prospective solution. The global, reliable and persistent nature of these sources makes them potentially able to guarantee the permanent monitoring of both coastal and open sea areas. To this aim, this thesis addresses the exploitation of Global Navigation Satellite Systems (GNSS) as transmitters of opportunity in passive bistatic radar (PBR) systems for maritime surveillance. The main limitation of this technology is the restricted power budget provided by navigation satellites, which makes it necessary to define innovative moving target detection techniques specifically tailored for the system under consideration. For this reason, this thesis puts forward long integration time techniques able to collect the signal energy over long time intervals (tens of seconds), allowing the retrieval of suitable levels of signal-to-disturbance ratios for detection purposes. The feasibility of this novel application is firstly investigated in a bistatic system configuration. A long integration time moving target detection technique working in bistatic range&Doppler plane is proposed and its effectiveness is proved against synthetic and experimental datasets. Subsequently the exploitation of multiple transmitters for the joint detection and localization of vessels at sea is also investigated. A single-stage approach to jointly detect and localize the ship targets by making use of long integration times (tens of seconds) and properly exploiting the spatial diversity offered by such a configuration is proposed. Furthermore, the potential of the system to extract information concerning the detected target characteristics for further target classification is assessed

Model-driven and Data-driven Approaches for some Object Recognition Problems

Recognizing objects from images and videos has been a long standing problem in computer vision. The recent surge in the prevalence of visual cameras has given rise to two main challenges where, (i) it is important to understand different sources of object variations in more unconstrained scenarios, and (ii) rather than describing an object in isolation, efficient learning methods for modeling object-scene `contextual' relations are required to resolve visual ambiguities. This dissertation addresses some aspects of these challenges, and consists of two parts. First part of the work focuses on obtaining object descriptors that are largely preserved across certain sources of variations, by utilizing models for image formation and local image features. Given a single instance of an object, we investigate the following three problems. (i) Representing a 2D projection of a 3D non-planar shape invariant to articulations, when there are no self-occlusions. We propose an articulation invariant distance that is preserved across piece-wise affine transformations of a non-rigid object `parts', under a weak perspective imaging model, and then obtain a shape context-like descriptor to perform recognition; (ii) Understanding the space of `arbitrary' blurred images of an object, by representing an unknown blur kernel of a known maximum size using a complete set of orthonormal basis functions spanning that space, and showing that subspaces resulting from convolving a clean object and its blurred versions with these basis functions are equal under some assumptions. We then view the invariant subspaces as points on a Grassmann manifold, and use statistical tools that account for the underlying non-Euclidean nature of the space of these invariants to perform recognition across blur; (iii) Analyzing the robustness of local feature descriptors to different illumination conditions. We perform an empirical study of these descriptors for the problem of face recognition under lighting change, and show that the direction of image gradient largely preserves object properties across varying lighting conditions. The second part of the dissertation utilizes information conveyed by large quantity of data to learn contextual information shared by an object (or an entity) with its surroundings. (i) We first consider a supervised two-class problem of detecting lane markings from road video sequences, where we learn relevant feature-level contextual information through a machine learning algorithm based on boosting. We then focus on unsupervised object classification scenarios where, (ii) we perform clustering using maximum margin principles, by deriving some basic properties on the affinity of `a pair of points' belonging to the same cluster using the information conveyed by `all' points in the system, and (iii) then consider correspondence-free adaptation of statistical classifiers across domain shifting transformations, by generating meaningful `intermediate domains' that incrementally convey potential information about the domain change

Introduction to Facial Micro Expressions Analysis Using Color and Depth Images: A Matlab Coding Approach (Second Edition, 2023)

The book attempts to introduce a gentle introduction to the field of Facial Micro Expressions Recognition (FMER) using Color and Depth images, with the aid of MATLAB programming environment. FMER is a subset of image processing and it is a multidisciplinary topic to analysis. So, it requires familiarity with other topics of Artifactual Intelligence (AI) such as machine learning, digital image processing, psychology and more. So, it is a great opportunity to write a book which covers all of these topics for beginner to professional readers in the field of AI and even without having background of AI. Our goal is to provide a standalone introduction in the field of MFER analysis in the form of theorical descriptions for readers with no background in image processing with reproducible Matlab practical examples. Also, we describe any basic definitions for FMER analysis and MATLAB library which is used in the text, that helps final reader to apply the experiments in the real-world applications. We believe that this book is suitable for students, researchers, and professionals alike, who need to develop practical skills, along with a basic understanding of the field. We expect that, after reading this book, the reader feels comfortable with different key stages such as color and depth image processing, color and depth image representation, classification, machine learning, facial micro-expressions recognition, feature extraction and dimensionality reduction. The book attempts to introduce a gentle introduction to the field of Facial Micro Expressions Recognition (FMER) using Color and Depth images, with the aid of MATLAB programming environment.Comment: This is the second edition of the boo