Application and evaluation of direct sparse visual odometry in marine vessels

With the international community pushing for a computer vision based option to the laws requiring a look-out for marine vehicles, there is now a significant motivation to provide digital solutions for navigation using these envisioned mandatory visual sensors. This paper explores the monocular direct sparse odometry algorithm when applied to a typical marine environment. The method uses a single camera to estimate a vessel\u27s motion and position over time and is then compared to ground truth to establish feasibility as both a local and global navigation system. Whilst it was inconsistent in accurately estimating vessel position, it was found that it could consistently estimate the vessel\u27s orientation in the majority of the situations the vessel was tasked with. It is therefore shown that monocular direct sparse odometry is partially suitable as a standalone navigation system and is a strong base for a multi-sensor solution

In-channel 3D models of riverine environments for hydromorphological characterization

Recent legislative approaches to improve the quality of rivers have resulted in the design and implementation of extensive and intensive monitoring programmes that are costly and time consuming. An important component of assessing the ecological status of a water body as required by the Water Framework Directive is characterising the hydromorphology. Recent advances in autonomous operation and the spatial coverage of monitoring systems enables more rapid 3D models of the river environment to be produced. This study presents a Structure from Motion (SfM) semi-autonomous based framework for the estimation of key reach hydromorphological measures such as water surface area, wetted water width, bank height, bank slope and bank-full width, using in-channel stereo-imagery. The framework relies on a stereo-camera that could be positioned on an autonomous boat. The proposed approach is demonstrated along three 40 m long reaches with differing hydromorphological characteristics. Results indicated that optimal stereo-camera settings need to be selected based on the river appearance. Results also indicated that the characteristics of the reach have an impact on the estimation of the hydromorphological measures; densely vegetated banks, presence of debris and sinuosity along the reach increased the overall error in hydromorphological measure estimation. The results obtained highlight a potential way forward towards the autonomous monitoring of freshwater ecosystem

Field-based measurement of hydrodynamics associated with engineered in-channel structures: the example of fish pass assessment

The construction of fish passes has been a longstanding measure to improve river ecosystem status by ensuring the passability of weirs, dams and other in- channel structures for migratory fish. Many fish passes have a low biological effectiveness because of unsuitable hydrodynamic conditions hindering fish to rapidly detect the pass entrance. There has been a need for techniques to quantify the hydrodynamics surrounding fish pass entrances in order to identify those passes that require enhancement and to improve the design of new passes. This PhD thesis presents the development of a methodology for the rapid, spatially continuous quantification of near-pass hydrodynamics in the field. The methodology involves moving-vessel Acoustic Doppler Current Profiler (ADCP) measurements in order to quantify the 3-dimensional water velocity distribution around fish pass entrances. The approach presented in this thesis is novel because it integrates a set of techniques to make ADCP data robust against errors associated with the environmental conditions near engineered in-channel structures. These techniques provide solutions to (i) ADCP compass errors from magnetic interference, (ii) bias in water velocity data caused by spatial flow heterogeneity, (iii) the accurate ADCP positioning in locales with constrained line of sight to navigation satellites, and (iv) the accurate and cost-effective sensor deployment following pre-defined sampling strategies. The effectiveness and transferability of the methodology were evaluated at three fish pass sites covering conditions of low, medium and high discharge. The methodology outputs enabled a detailed quantitative characterisation of the fish pass attraction flow and its interaction with other hydrodynamic features. The outputs are suitable to formulate novel indicators of hydrodynamic fish pass attractiveness and they revealed the need to refine traditional fish pass design guidelines

Real-time classification of vehicle types within infra-red imagery.

Real-time classification of vehicles into sub-category types poses a significant challenge within infra-red imagery due to the high levels of intra-class variation in thermal vehicle signatures caused by aspects of design, current operating duration and ambient thermal conditions. Despite these challenges, infra-red sensing offers significant generalized target object detection advantages in terms of all-weather operation and invariance to visual camouflage techniques. This work investigates the accuracy of a number of real-time object classification approaches for this task within the wider context of an existing initial object detection and tracking framework. Specifically we evaluate the use of traditional feature-driven bag of visual words and histogram of oriented gradient classification approaches against modern convolutional neural network architectures. Furthermore, we use classical photogrammetry, within the context of current target detection and classification techniques, as a means of approximating 3D target position within the scene based on this vehicle type classification. Based on photogrammetric estimation of target position, we then illustrate the use of regular Kalman filter based tracking operating on actual 3D vehicle trajectories. Results are presented using a conventional thermal-band infra-red (IR) sensor arrangement where targets are tracked over a range of evaluation scenarios

Journal of Applied Hydrography

Fokusthema: Fernerkundung und Laserbathymetri

Visual localization in challenging environments

Visual localization, the method of self-localization based on camera images, has established as an additional, GNSS-free technology that is investigated in increasingly real and challenging applications. Particularly demanding is the self-localization of first responders in unstructured and unknown environments, for which visual localization can substantially contribute to increase the situational awareness and safety of first responders. Challenges arise from the operation under adverse conditions on computationally restricted platforms in the presence of dynamic objects. Current solutions are quickly pushed to their limits and the development of more robust approaches is of high demand. This thesis investigates the application of visual localization in dynamic, adverse environments to identify challenges and accordingly to increase the robustness, on the example of a dedicated visual-inertial navigation system. The methodical contributions of this work relate to the introduction of semantic understanding, improvements in error propagation and the development of a digital twin. The geometric visual odometry component is extended to a hybrid approach that includes a deep neural network for semantic segmentation to ignore distracting image areas of certain object classes. A Sensor-AI approach complements this method by directly training the network to segment image areas that are critical for the considered visual odometry system. Another improvement results from analyses and modifications of the existing error propagation in visual odometry. Furthermore, a digital twin is presented that closely replicates geometric and radiometric properties of the real sensor system in simulation in order to multiply experimental possibilities. The experiments are based on datasets from inspections that are used to motivate three first responder scenarios, namely indoor rescue, flood disaster and wildfire. The datasets were recorded in corridor, mall, coast, river and fumarole environments and aim to analyze the influence of the dynamic elements person, water and smoke. Each investigation starts with extensive in-depth analyses in simulation based on created synthetic video clones of the respective dynamic environments. Specifically, a combined sensitivity analysis allows to jointly consider environment, system design, sensor property and calibration error parameters to account for adverse conditions. All investigations are verified with experiments based on the real system. The results show the susceptibility of geometric approaches to dynamic objects in challenging scenarios. The introduction of the segmentation aid within the hybrid system contributes well in terms of robustness by preventing significant failures, but understandably it cannot compensate for a lack of visible static backgrounds. As a consequence, future visual localization systems require both the ability of semantic understanding and its integration into a complementary multi-sensor system

Quantitative Evaluation of Stereo Visual Odometry for Autonomous Vessel Localisation in Inland Waterway Sensing Applications

Autonomous survey vessels can increase the efficiency and availability of wide-area river environment surveying as a tool for environment protection and conservation. A key challenge is the accurate localisation of the vessel, where bank-side vegetation or urban settlement preclude the conventional use of line-of-sight global navigation satellite systems (GNSS). In this paper, we evaluate unaided visual odometry, via an on-board stereo camera rig attached to the survey vessel, as a novel, low-cost localisation strategy. Feature-based and appearance-based visual odometry algorithms are implemented on a six degrees of freedom platform operating under guided motion, but stochastic variation in yaw, pitch and roll. Evaluation is based on a 663 m-long trajectory (>15,000 image frames) and statistical error analysis against ground truth position from a target tracking tachymeter integrating electronic distance and angular measurements. The position error of the feature-based technique (mean of ±0.067 m) is three times smaller than that of the appearance-based algorithm. From multi-variable statistical regression, we are able to attribute this error to the depth of tracked features from the camera in the scene and variations in platform yaw. Our findings inform effective strategies to enhance stereo visual localisation for the specific application of river monitoring

Undergraduate engineering and built environment project conference 2017: book of abstracts - Toowoomba, Australia, 18-22 September 2017

Book of Abstracts of the USQ Undergraduate Engineering and Built Environment Conference 2017, held Toowoomba, Australia, 18-22 September 2017. These proceedings include extended abstracts of the verbal presentations that are delivered at the project conference. The work reported at the conference is the research undertaken by students in meeting the requirements of courses ENG4111/ENG4112 Research Project