Hybrid Marker-less Camera Pose Tracking with Integrated Sensor Fusion

This thesis presents a framework for a hybrid model-free marker-less inertial-visual camera pose tracking with an integrated sensor fusion mechanism. The proposed solution addresses the fundamental problem of pose recovery in computer vision and robotics and provides an improved solution for wide-area pose tracking that can be used on mobile platforms and in real-time applications. In order to arrive at a suitable pose tracking algorithm, an in-depth investigation was conducted into current methods and sensors used for pose tracking. Preliminary experiments were then carried out on hybrid GPS-Visual as well as wireless micro-location tracking in order to evaluate their suitability for camera tracking in wide-area or GPS-denied environments. As a result of this investigation a combination of an inertial measurement unit and a camera was chosen as the primary sensory inputs for a hybrid camera tracking system. After following a thorough modelling and mathematical formulation process, a novel and improved hybrid tracking framework was designed, developed and evaluated. The resulting system incorporates an inertial system, a vision-based system and a recursive particle filtering-based stochastic data fusion and state estimation algorithm. The core of the algorithm is a state-space model for motion kinematics which, combined with the principles of multi-view camera geometry and the properties of optical flow and focus of expansion, form the main components of the proposed framework. The proposed solution incorporates a monitoring system, which decides on the best method of tracking at any given time based on the reliability of the fresh vision data provided by the vision-based system, and automatically switches between visual and inertial tracking as and when necessary. The system also includes a novel and effective self-adjusting mechanism, which detects when the newly captured sensory data can be reliably used to correct the past pose estimates. The corrected state is then propagated through to the current time in order to prevent sudden pose estimation errors manifesting as a permanent drift in the tracking output. Following the design stage, the complete system was fully developed and then evaluated using both synthetic and real data. The outcome shows an improved performance compared to existing techniques, such as PTAM and SLAM. The low computational cost of the algorithm enables its application on mobile devices, while the integrated self-monitoring, self-adjusting mechanisms allow for its potential use in wide-area tracking applications

Unmanned aerial vehicle communications for civil applications: a review

The use of drones, formally known as unmanned aerial vehicles (UAVs), has significantly increased across a variety of applications over the past few years. This is due to the rapid advancement towards the design and production of inexpensive and dependable UAVs and the growing request for the utilization of such platforms particularly in civil applications. With their intrinsic attributes such as high mobility, rapid deployment and flexible altitude, UAVs have the potential to be utilized in many wireless system applications. On the one hand, UAVs are able to operate as flying mobile terminals within wireless/cellular networks to support a variety of missions such as goods delivery, search and rescue, precision agriculture monitoring, and remote sensing. On the other hand, UAVs can be utilized as aerial base stations to increase wireless communication coverage, reliability, and the capacity of wireless systems without additional investment in wireless systems infrastructure. The aim of this article is to review the current applications of UAVs for civil and commercial purposes. The focus of this paper is on the challenges and communication requirements associated with UAV-based communication systems. This article initially classifies UAVs in terms of various parameters, some of which can impact UAVs’ communication performance. It then provides an overview of aerial networking and investigates UAVs routing protocols specifically, which are considered as one of the challenges in UAV communication. This article later investigates the use of UAV networks in a variety of civil applications and considers many challenges and communication demands of these applications. Subsequently, different types of simulation platforms are investigated from a communication and networking viewpoint. Finally, it identifies areas of future research

Visual and Camera Sensors

This book includes 13 papers published in Special Issue ("Visual and Camera Sensors") of the journal Sensors. The goal of this Special Issue was to invite high-quality, state-of-the-art research papers dealing with challenging issues in visual and camera sensors

Proceedings of the 2021 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

2021, the annual joint workshop of the Fraunhofer IOSB and KIT IES was hosted at the IOSB in Karlsruhe. For a week from the 2nd to the 6th July the doctoral students extensive reports on the status of their research. The results and ideas presented at the workshop are collected in this book in the form of detailed technical reports

Proceedings of the 2021 Joint Workshop of Fraunhofer IOSB and Institute for Anthropomatics, Vision and Fusion Laboratory

2021, the annual joint workshop of the Fraunhofer IOSB and KIT IES was hosted at the IOSB in Karlsruhe. For a week from the 2nd to the 6th July the doctoral students extensive reports on the status of their research. The results and ideas presented at the workshop are collected in this book in the form of detailed technical reports

SLIM-A Scalable and Lightweight Indoor-Navigation MAV as Research and Education Platform

Indoor navigation with micro aerial vehicles (MAVs) is of growing importance nowadays. State of the art flight management controllers provide extensive interfaces for control and navigation, but most commonly aim for performing in outdoor navigation scenarios. Indoor navigation with MAVs is challenging, because of spatial constraints and lack of drift-free positioning systems like GPS. Instead, vision and/or inertial-based methods are used to localize the MAV against the environment. For educational purposes and moreover to test and develop such algorithms, since 2015 the so called droneSpace was established at the Institute of Computer Graphics and Vision at Graz University of Technology. It consists of a flight arena which is equipped with a highly accurate motion tracking system and further holds an extensive robotics framework for semi-autonomous MAV navigation. A core component of the droneSpace is a Scalable and Lightweight Indoor-navigation MAV design, which we call the SLIM (A detailed description of the SLIM and related projects can be found at our website: https://sites.google.com/view/w-a-isop/home/education/slim). It allows flexible vision-sensor setups and moreover provides interfaces to inject accurate pose measurements form external tracking sources to achieve stable indoor hover-flights. With this work we present capabilities of the framework and its flexibility, especially with regards to research and education at university level. We present use cases from research projects but also courses at the Graz University of Technology, whereas we discuss results and potential future work on the platform

A multidisciplinary framework for mission effectiveness quantification and assessment of micro autonomous systems and technologies

Micro Autonomous Systems and Technologies (MAST) is an Army Research Laboratory (ARL) sponsored project based on a consortium of revolutionary academic and industrial research institutions working together to develop new technologies in the field of microelectronics, autonomy, micromechanics and integration. The overarching goal of the MAST consortium is to develop autonomous, multifunctional, and collaborative ensembles of microsystems to enhance small unit tactical situational awareness in urban and complex terrain. Unmanned systems are used to obtain intelligence at the macro level, but there is no real-time intelligence asset at the squad level. MAST seeks to provide that asset. Consequently, multiple integrated MAST heterogeneous platforms (e.g. crawlers, flyers, etc.) working together synergistically as an ensemble shall be capable of autonomously performing a wide spectrum of operational functions based on the latest developments in micro-mechanics, micro-electronics, and power technologies to achieve the desired operational objectives. The design of such vehicles is, by nature, highly constrained in terms of size, weight and power. Technologists are trying to understand the impacts of developing state-of-the-art technologies on the MAST systems while the operators are trying to define strategies and tactics on how to use these systems. These two different perspectives create an integration gap. The operators understand the capabilities needed on the field of deployment but not necessarily the technologies, while the technologists understand the physics of the technologies but not necessarily how they will be deployed, utilized, and operated during a mission. This not only results in a major requirements disconnect, representing the difference of perspectives between soldiers and the researchers, but also demonstrates the lack of quantified means to assess the technology gap in terms of mission requirements. This necessitates the quantification and resolution of the requirements disconnect and technology gap leading to re-definitions of the requirements based on mission scenarios. A research plan, built on a technical approach based on the simultaneous application of decomposition and re-composition or 'Top-down' and 'Bottom-up' approaches, was used for development of a structured and traceable methodology. The developed methodology is implemented through an integrated framework consisting of various decision-making tools, modeling and simulation, and experimental data farming and validation. The major obstacles in the development of the presented framework stemmed from the fact that all MAST technologies are revolutionary in nature, with no available historical data, sizing and synthesis codes or reliable physics-based models. The inherently multidisciplinary, multi-objective and uncertain nature of MAST technologies makes it very difficult to map mission level objectives to measurable engineering metrics. It involves the optimization of multiple disciplines such as Aero, CS/CE, ME, EE, Biology, etc., and of multiple objectives such as mission performance, tactics, vehicle attributes, etc. Furthermore, the concept space is enormous with hundreds of billions of alternatives, and largely includes future technologies with low Technology Readiness Level (TRL) resulting in high uncertainty. The presented framework is a cyber-physical design and analysis suite that combines Warfighter mission needs and expert technologist knowledge with a set of design and optimization tools, models, and experiments in order to provide a quantitative measure of the requirements disconnect and technology gap mentioned above. This quantification provides the basis for re-definitions of the requirements that are realistic in nature and ensure mission success. The research presents the development of this methodology and framework to address the core research objectives. The developed framework was then implemented on two mission scenarios that are of interest to the MAST consortium and Army Research Laboratory, namely, Joppa Urban Dwelling and Black Hawk Down Interior Building Reconnaissance. Results demonstrate the framework’s validity and serve as proof of concept for bridging the requirements disconnect between the Warfighter and the technologists. Billions of alternative MAST vehicles, composed of current and future technologies, were modeled and simulated, as part of a swarm, to evaluate their mission performance. In-depth analyses of the experiments, conducted as part of the research, presents quantitative technology gaps that needs to be addressed by technologist for successful mission completion. Quantitative values for vehicle specifications and systems' Measures of Performance were determined for acceptable level of performance for the given missions. The consolidated results were used for defining mission based requirements of MAST systems.Ph.D

Bio-Inspired Robotics

Modern robotic technologies have enabled robots to operate in a variety of unstructured and dynamically-changing environments, in addition to traditional structured environments. Robots have, thus, become an important element in our everyday lives. One key approach to develop such intelligent and autonomous robots is to draw inspiration from biological systems. Biological structure, mechanisms, and underlying principles have the potential to provide new ideas to support the improvement of conventional robotic designs and control. Such biological principles usually originate from animal or even plant models, for robots, which can sense, think, walk, swim, crawl, jump or even fly. Thus, it is believed that these bio-inspired methods are becoming increasingly important in the face of complex applications. Bio-inspired robotics is leading to the study of innovative structures and computing with sensory–motor coordination and learning to achieve intelligence, flexibility, stability, and adaptation for emergent robotic applications, such as manipulation, learning, and control. This Special Issue invites original papers of innovative ideas and concepts, new discoveries and improvements, and novel applications and business models relevant to the selected topics of ``Bio-Inspired Robotics''. Bio-Inspired Robotics is a broad topic and an ongoing expanding field. This Special Issue collates 30 papers that address some of the important challenges and opportunities in this broad and expanding field

Intelligent Biosignal Processing in Wearable and Implantable Sensors

This reprint provides a collection of papers illustrating the state-of-the-art of smart processing of data coming from wearable, implantable or portable sensors. Each paper presents the design, databases used, methodological background, obtained results, and their interpretation for biomedical applications. Revealing examples are brain–machine interfaces for medical rehabilitation, the evaluation of sympathetic nerve activity, a novel automated diagnostic tool based on ECG data to diagnose COVID-19, machine learning-based hypertension risk assessment by means of photoplethysmography and electrocardiography signals, Parkinsonian gait assessment using machine learning tools, thorough analysis of compressive sensing of ECG signals, development of a nanotechnology application for decoding vagus-nerve activity, detection of liver dysfunction using a wearable electronic nose system, prosthetic hand control using surface electromyography, epileptic seizure detection using a CNN, and premature ventricular contraction detection using deep metric learning. Thus, this reprint presents significant clinical applications as well as valuable new research issues, providing current illustrations of this new field of research by addressing the promises, challenges, and hurdles associated with the synergy of biosignal processing and AI through 16 different pertinent studies. Covering a wide range of research and application areas, this book is an excellent resource for researchers, physicians, academics, and PhD or master students working on (bio)signal and image processing, AI, biomaterials, biomechanics, and biotechnology with applications in medicine

2015, UMaine News Press Releases

This is a catalog of press releases put out by the University of Maine Division of Marketing and Communications between January 2, 2015 and December 31, 2015