Real-time infinite horizon tracking with data fusion for augmented reality in a maritime operations context.

10 pagesInternational audienceIn this paper, we propose a method for real-time horizon tracking (i.e., separation line between the sky and the sea) in a maritime operations context. We present the fusion of an image processing algorithm with the data obtained from the inertial measurement unit (IMU). The initial aim is to filter out environmental conditions using inertial information in order to combine a video stream with onboard electronic charts. This is achieved by the detection of the horizon with an image processing algorithm in an area defined by the IMU. We then present an evaluation of the algorithm with regard to the rate of detection of the horizon and the impact of the image resolution on the computational time. The purpose of developing this method is to create an augmented reality maritime operations application. We combine the video stream with electronic charts in a single display. We use the position of the horizon in the image to split the display into different areas. Then, we use transparency to display the video, the electronic charts or both

A review of augmented reality applications for ship bridges

We present a state-of-the art analysis of Augmented Reality (AR) applications for ship bridge operation. We compiled and reviewed what type of use cases were published, what type of maritime applications have been adapted to AR, how they were prototyped and evaluated and what type of technology was used. We also reviewed the user interaction mechanisms, information display and adaptation to maritime environmental conditions. Our analysis shows that although there are many examples of AR applications in ship bridges, there is still much work that needs to be done before these solutions can be suitably adapted to commercial settings. In addition, we argue there is a need to develop design requirements and regulations that can guide the safe development of AR

DOES: A Deep Learning-based approach to estimate roll and pitch at sea

The use of Attitude and Heading Reference Systems (AHRS) for orientation estimation is now common practice in a wide range of applications, e.g., robotics and human motion tracking, aerial vehicles and aerospace, gaming and virtual reality, indoor pedestrian navigation and maritime navigation. The integration of the high-rate measurements can provide very accurate estimates, but these can suffer from errors accumulation due to the sensors drift over longer time scales. To overcome this issue, inertial sensors are typically combined with additional sensors and techniques. As an example, camera-based solutions have drawn a large attention by the community, thanks to their low-costs and easy hardware setup; moreover, impressive results have been demonstrated in the context of Deep Learning. This work presents the preliminary results obtained by DOES, a supportive Deep Learning method specifically designed for maritime navigation, which aims at improving the roll and pitch estimations obtained by common AHRS. DOES recovers these estimations through the analysis of the frames acquired by a low-cost camera pointing the horizon at sea. The training has been performed on the novel ROPIS dataset, presented in the context of this work, acquired using the FrameWO application developed for the scope. Promising results encourage to test other network backbones and to further expand the dataset, improving the accuracy of the results and the range of applications of the method as a valid support to visual-based odometry techniques

Proceedings of the 2011 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

This book is a collection of 15 reviewed technical reports summarizing the presentations at the 2011 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory. The covered topics include image processing, optical signal processing, visual inspection, pattern recognition and classification, human-machine interaction, world and situation modeling, autonomous system localization and mapping, information fusion, and trust propagation in sensor networks

Multiple-Aspect Analysis of Semantic Trajectories

This open access book constitutes the refereed post-conference proceedings of the First International Workshop on Multiple-Aspect Analysis of Semantic Trajectories, MASTER 2019, held in conjunction with the 19th European Conference on Machine Learning and Knowledge Discovery in Databases, ECML PKDD 2019, in Würzburg, Germany, in September 2019. The 8 full papers presented were carefully reviewed and selected from 12 submissions. They represent an interesting mix of techniques to solve recurrent as well as new problems in the semantic trajectory domain, such as data representation models, data management systems, machine learning approaches for anomaly detection, and common pathways identification

Feature-based object tracking in maritime scenes.

A monitoring of presence, location and activity of various objects on the sea is essential for maritime navigation and collision avoidance. Mariners normally rely on two complementary methods of the monitoring: radar and satellite-based aids and human observation. Though radar aids are relatively accurate at long distances, their capability of detecting small, unmanned or non-metallic craft that generally do not reflect radar waves sufficiently enough, is limited. The mariners, therefore, rely in such cases on visual observations. The visual observation is often facilitated by using cameras overlooking the sea that can also provide intensified infra-red images. These systems or nevertheless merely enhance the image and the burden of the tedious and error-prone monitoring task still rests with the operator. This thesis addresses the drawbacks of both methods by presenting a framework consisting of a set of machine vision algorithms that facilitate the monitoring tasks in maritime environment. The framework detects and tracks objects in a sequence of images captured by a camera mounted either on a board of a vessel or on a static platform over-looking the sea. The detection of objects is independent of their appearance and conditions such as weather and time of the day. The output of the framework consists of locations and motions of all detected objects with respect to a fixed point in the scene. All values are estimated in real-world units, i. e. location is expressed in metres and velocity in knots. The consistency of the estimates is maintained by compensating for spurious effects such as vibration of the camera. In addition, the framework continuously checks for predefined events such as collision threats or area intrusions, raising an alarm when any such event occurs. The development and evaluation of the framework is based on sequences captured under conditions corresponding to a designated application. The independence of the detection and tracking on the appearance of the sceneand objects is confirmed by a final cross-validation of the framework on previously unused sequences. Potential applications of the framework in various areas of maritime environment including navigation, security, surveillance and others are outlined. Limitations to the presented framework are identified and possible solutions suggested. The thesis concludes with suggestions to further directions of the research presented

Multiple-vehicle resource-constrained navigation in the deep ocean

Thesis (S.M.)--Joint Program in Applied Ocean Science and Engineering (Massachusetts Institute of Technology, Dept. of Mechanical Engineering; and the Woods Hole Oceanographic Institution), 2011.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Cataloged from student submitted PDF version of thesis.Includes bibliographical references (p. 139-148).This thesis discusses sensor management methods for multiple-vehicle fleets of autonomous underwater vehicles, which will allow for more efficient and capable infrastructure in marine science, industry, and naval applications. Navigation for fleets of vehicles in the ocean presents a large challenge, as GPS is not available underwater and dead-reckoning based on inertial or bottom-lock methods can require expensive sensors and suffers from drift. Due to zero drift, acoustic navigation methods are attractive as replacements or supplements to dead-reckoning, and centralized systems such as an Ultra-Short Baseline Sonar (USBL) allow for small and economical components onboard the individual vehicles. Motivated by subsea equipment delivery, we present model-scale proof-of-concept experimental pool tests of a prototype Vertical Glider Robot (VGR), a vehicle designed for such a system. Due to fundamental physical limitations of the underwater acoustic channel, a sensor such as the USBL is limited in its ability to track multiple targets-at best a small subset of the entire fleet may be observed at once, at a low update rate. Navigation updates are thus a limited resource and must be efficiently allocated amongst the fleet in a manner that balances the exploration versus exploitation tradeoff. The multiple vehicle tracking problem is formulated in the Restless Multi-Armed Bandit structure following the approach of Whittle in [108], and we investigate in detail the Restless Bandit Kalman Filters priority index algorithm given by Le Ny et al. in [71]. We compare round-robin and greedy heuristic approaches with the Restless Bandit approach in computational experiments. For the subsea equipment delivery example of homogeneous vehicles with depth-varying parameters, a suboptimal quasi-static approximation of the index algorithm balances low landing error with safety and robustness. For infinite-horizon tracking of systems with linear time-invariant parameters, the index algorithm is optimal and provides benefits of up to 40% over the greedy heuristic for heterogeneous vehicle fleets. The index algorithm can match the performance of the greedy heuristic for short horizons, and offers the greatest improvement for long missions, when the infinite-horizon assumption is reasonably met.by Brooks Louis-Kiguchi Reed.S.M

Proceedings, MSVSCC 2019

Old Dominion University Department of Modeling, Simulation & Visualization Engineering (MSVE) and the Virginia Modeling, Analysis and Simulation Center (VMASC) held the 13th annual Modeling, Simulation & Visualization (MSV) Student Capstone Conference on April 18, 2019. The Conference featured student research and student projects that are central to MSV. Also participating in the conference were faculty members who volunteered their time to impart direct support to their students’ research, facilitated the various conference tracks, served as judges for each of the tracks, and provided overall assistance to the conference. Appreciating the purpose of the conference and working in a cohesive, collaborative effort, resulted in a successful symposium for everyone involved. These proceedings feature the works that were presented at the conference. Capstone Conference Chair: Dr. Yuzhong Shen Capstone Conference Student Chair: Daniel Pere

Deep Learning Based Malware Classification Using Deep Residual Network

The traditional malware detection approaches rely heavily on feature extraction procedure, in this paper we proposed a deep learning-based malware classification model by using a 18-layers deep residual network. Our model uses the raw bytecodes data of malware samples, converting the bytecodes to 3-channel RGB images and then applying the deep learning techniques to classify the malwares. Our experiment results show that the deep residual network model achieved an average accuracy of 86.54% by 5-fold cross validation. Comparing to the traditional methods for malware classification, our deep residual network model greatly simplify the malware detection and classification procedures, it achieved a very good classification accuracy as well. The dataset we used in this paper for training and testing is Malimg dataset, one of the biggest malware datasets released by vision research lab of UCSB