In-Flight Wing Deformation Measurement System for Small Unmanned Aerial Vehicles

Peer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/140711/1/6.2014-0330.pd

Developing a person guidance module for hospital robots

This dissertation describes the design and implementation of the Person Guidance Module (PGM) that enables the IWARD (Intelligent Robot Swarm for attendance, Recognition, Cleaning and delivery) base robot to offer route guidance service to the patients or visitors inside the hospital arena. One of the common problems encountered in huge hospital buildings today is foreigners not being able to find their way around in the hospital. Although there are a variety of guide robots currently existing on the market and offering a wide range of guidance and related activities, they do not fit into the modular concept of the IWARD project. The PGM features a robust and foolproof non-hierarchical sensor fusion approach of an active RFID, stereovision and cricket mote sensor for guiding a patient to the X-ray room, or a visitor to a patient’s ward in every possible scenario in a complex, dynamic and crowded hospital environment. Moreover, the speed of the robot can be adjusted automatically according to the pace of the follower for physical comfort using this system. Furthermore, the module performs these tasks in any unconstructed environment solely from a robot’s onboard perceptual resources in order to limit the hardware installation costs and therefore the indoor setting support. Similar comprehensive solution in one single platform has remained elusive in existing literature. The finished module can be connected to any IWARD base robot using quick-change mechanical connections and standard electrical connections. The PGM module box is equipped with a Gumstix embedded computer for all module computing which is powered up automatically once the module box is inserted into the robot. In line with the general software architecture of the IWARD project, all software modules are developed as Orca2 components and cross-complied for Gumstix’s XScale processor. To support standardized communication between different software components, Internet Communications Engine (Ice) has been used as middleware. Additionally, plug-and-play capabilities have been developed and incorporated so that swarm system is aware at all times of which robot is equipped with PGM. Finally, in several field trials in hospital environments, the person guidance module has shown its suitability for a challenging real-world application as well as the necessary user acceptance

VIRTUAL SURFACE FOR HUMAN-ROBOT INTERACTION

As cooperation between robots and humans becomes increasingly important for new robotic applications, human-robot interaction (HRI) becomes a significant area of research. This paper presents a novel approach to HRI based on the use of a virtual surface. The presented system consists of a virtual surface and a robot manipulator capable of tactile interaction. Multimedia content of the virtual surface and the option to manually guide the manipulator through space provide an intuitive means of interaction between the robot and the operator. The paper proposes shared workspaces for humans and robots to simplify and improve human-robot collaboration when performing various tasks utilizing a developed interaction model

Automated location of active fire perimeters in aerial infrared imaging using unsupervised edge detectors

A variety of remote sensing techniques have been applied to forest fires. However, there is at present no system capable of monitoring an active fire precisely in a totally automated manner. Spaceborne sensors show too coarse spatio-temporal resolutions and all previous studies that extracted fire properties from infrared aerial imagery incorporated manual tasks within the image processing workflow. As a contribution to this topic, this paper presents an algorithm to automatically locate the fuel burning interface of an active wildfire in georeferenced aerial thermal infrared (TIR) imagery. An unsupervised edge detector, built upon the Canny method, was accompanied by the necessary modules for the extraction of line coordinates and the location of the total burned perimeter. The system was validated in different scenarios ranging from laboratory tests to large-scale experimental burns performed under extreme weather conditions. Output accuracy was computed through three common similarity indices and proved acceptable. Computing times were below 1¿s per image on average. The produced information was used to measure the temporal evolution of the fire perimeter and automatically generate rate of spread (ROS) fields. Information products were easily exported to standard Geographic Information Systems (GIS), such as GoogleEarth and QGIS. Therefore, this work contributes towards the development of an affordable and totally automated system for operational wildfire surveillance.Peer ReviewedPostprint (author's final draft

3D VISUAL TRACKING USING A SINGLE CAMERA

automated surveillance and motion based recognition. 3D tracking address the localization of moving target is the 3D space. Therefore, 3D tracking requires 3D measurement of the moving object which cannot be obtained from 2D cameras. Existing 3D tracking systems use multiple cameras for computing the depth of field and it is only used in research laboratories. Millions of surveillance cameras are installed worldwide and all of them capture 2D images. Therefore, 3D tracking cannot be performed with these cameras unless multiple cameras are installed at each location in order to compute the depth. This means installing millions of new cameras which is not a feasible solution. This work introduces a novel depth estimation method from a single 2D image using triangulation. This method computes the absolute depth of field for any object in the scene with high accuracy and short computational time. The developed method is used for performing 3D visual tracking using a single camera by providing the depth of field and ground coordinates of the moving object for each frame accurately and efficiently. Therefore, this technique can help in transforming existing 2D tracking and 2D video analytics into 3D without incurring additional costs. This makes video surveillance more efficient and increases its usage in human life. The proposed methodology uses background subtraction process for detecting a moving object in the image. Then, the newly developed depth estimation method is used for computing the 3D measurement of the moving target. Finally, the unscented Kalman filter is used for tracking the moving object given the 3D measurement obtained by the triangulation method. This system has been test and validated using several video sequences and it shows good performance in term of accuracy and computational complexity

Object distance measurement using a single camera for robotic applications

Visual servoing is defined as controlling robots by extracting data obtained from the vision system, such as the distance of an object with respect to a reference frame, or the length and width of the object. There are three image-based object distance measurement techniques: i) using two cameras, i.e., stereovision; ii) using a single camera, i.e., monovision; and iii) time-of-flight camera. The stereovision method uses two cameras to find the object’s depth and is highly accurate. However, it is costly compared to the monovision technique due to the higher computational burden and the cost of two cameras (rather than one) and related accessories. In addition, in stereovision, a larger number of images of the object need to be processed in real-time, and by increasing the distance of the object from cameras, the measurement accuracy decreases. In the time-of-flight distance measurement technique, distance information is obtained by measuring the total time for the light to transmit to and reflect from the object. The shortcoming of this technique is that it is difficult to separate the incoming signal, since it depends on many parameters such as the intensity of the reflected light, the intensity of the background light, and the dynamic range of the sensor. However, for applications such as rescue robot or object manipulation by a robot in a home and office environment, the high accuracy distance measurement provided by stereovision is not required. Instead, the monovision approach is attractive for some applications due to: i) lower cost and lower computational burden; and ii) lower complexity due to the use of only one camera. Using a single camera for distance measurement, object detection and feature extraction (i.e., finding the length and width of an object) is not yet well researched and there are very few published works on the topic in the literature. Therefore, using this technique for real-world robotics applications requires more research and improvements. This thesis mainly focuses on the development of object distance measurement and feature extraction algorithms using a single fixed camera and a single camera with variable pitch angle based on image processing techniques. As a result, two different improved and modified object distance measurement algorithms were proposed for cases where a camera is fixed at a given angle in the vertical plane and when it is rotating in a vertical plane. In the proposed algorithms, as a first step, the object distance and dimension such as length and width were obtained using existing image processing techniques. Since the results were not accurate due to lens distortion, noise, variable light intensity and other uncertainties such as deviation of the position of the object from the optical axes of camera, in the second step, the distance and dimension of the object obtained from existing techniques were modified in the X- and Y-directions and for the orientation of the object about the Z-axis in the object plane by using experimental data and identification techniques such as the least square method. Extensive experimental results confirmed that the accuracy increased for measured distance from 9.4 mm to 2.95 mm, for length from 11.6 mm to 2.2 mm, and for width from 18.6 mm to 10.8 mm. In addition, the proposed algorithm is significantly improved with proposed corrections compared to existing methods. Furthermore, the improved distance measurement method is computationally efficient and can be used for real-time robotic application tasks such as pick and place and object manipulation in a home or office environment.Master's Thesi

Real-time on-board obstacle avoidance for UAVs based on embedded stereo vision

In order to improve usability and safety, modern unmanned aerial vehicles (UAVs) are equipped with sensors to monitor the environment, such as laser-scanners and cameras. One important aspect in this monitoring process is to detect obstacles in the flight path in order to avoid collisions. Since a large number of consumer UAVs suffer from tight weight and power constraints, our work focuses on obstacle avoidance based on a lightweight stereo camera setup. We use disparity maps, which are computed from the camera images, to locate obstacles and to automatically steer the UAV around them. For disparity map computation we optimize the well-known semi-global matching (SGM) approach for the deployment on an embedded FPGA. The disparity maps are then converted into simpler representations, the so called U-/V-Maps, which are used for obstacle detection. Obstacle avoidance is based on a reactive approach which finds the shortest path around the obstacles as soon as they have a critical distance to the UAV. One of the fundamental goals of our work was the reduction of development costs by closing the gap between application development and hardware optimization. Hence, we aimed at using high-level synthesis (HLS) for porting our algorithms, which are written in C/C++, to the embedded FPGA. We evaluated our implementation of the disparity estimation on the KITTI Stereo 2015 benchmark. The integrity of the overall realtime reactive obstacle avoidance algorithm has been evaluated by using Hardware-in-the-Loop testing in conjunction with two flight simulators.Comment: Accepted in the International Archives of the Photogrammetry, Remote Sensing and Spatial Information Scienc

INCORPORATING MACHINE VISION IN PRECISION DAIRY FARMING TECHNOLOGIES

The inclusion of precision dairy farming technologies in dairy operations is an area of increasing research and industry direction. Machine vision based systems are suitable for the dairy environment as they do not inhibit workflow, are capable of continuous operation, and can be fully automated. The research of this dissertation developed and tested 3 machine vision based precision dairy farming technologies tailored to the latest generation of RGB+D cameras. The first system focused on testing various imaging approaches for the potential use of machine vision for automated dairy cow feed intake monitoring. The second system focused on monitoring the gradual change in body condition score (BCS) for 116 cows over a nearly 7 month period. Several proposed automated BCS systems have been previously developed by researchers, but none have monitored the gradual change in BCS for a duration of this magnitude. These gradual changes infer a great deal of beneficial and immediate information on the health condition of every individual cow being monitored. The third system focused on automated dairy cow feature detection using Haar cascade classifiers to detect anatomical features. These features included the tailhead, hips, and rear regions of the cow body. The features chosen were done so in order to aid machine vision applications in determining if and where a cow is present in an image or video frame. Once the cow has been detected, it must then be automatically identified in order to keep the system fully automated, which was also studied in a machine vision based approach in this research as a complimentary aspect to incorporate along with cow detection. Such systems have the potential to catch poor health conditions developing early on, aid in balancing the diet of the individual cow, and help farm management to better facilitate resources, monetary and otherwise, in an appropriate and efficient manner. Several different applications of this research are also discussed along with future directions for research, including the potential for additional automated precision dairy farming technologies, integrating many of these technologies into a unified system, and the use of alternative, potentially more robust machine vision cameras

An investigation into common challenges of 3D scene understanding in visual surveillance

Nowadays, video surveillance systems are ubiquitous. Most installations simply consist of CCTV cameras connected to a central control room and rely on human operators to interpret what they see on the screen in order to, for example, detect a crime (either during or after an event). Some modern computer vision systems aim to automate the process, at least to some degree, and various algorithms have been somewhat successful in certain limited areas. However, such systems remain inefficient in general circumstances and present real challenges yet to be solved. These challenges include the ability to recognise and ultimately predict and prevent abnormal behaviour or even reliably recognise objects, for example in order to detect left luggage or suspicious objects. This thesis first aims to study the state-of-the-art and identify the major challenges and possible requirements of future automated and semi-automated CCTV technology in the field. This thesis presents the application of a suite of 2D and highly novel 3D methodologies that go some way to overcome current limitations.The methods presented here are based on the analysis of object features directly extracted from the geometry of the scene and start with a consideration of mainly existing techniques, such as the use of lines, vanishing points (VPs) and planes, applied to real scenes. Then, an investigation is presented into the use of richer 2.5D/3D surface normal data. In all cases the aim is to combine both 2D and 3D data to obtain a better understanding of the scene, aimed ultimately at capturing what is happening within the scene in order to be able to move towards automated scene analysis. Although this thesis focuses on the widespread application of video surveillance, an example case of the railway station environment is used to represent typical real-world challenges, where the principles can be readily extended elsewhere, such as to airports, motorways, the households, shopping malls etc. The context of this research work, together with an overall presentation of existing methods used in video surveillance and their challenges are described in chapter 1.Common computer vision techniques such as VP detection, camera calibration, 3D reconstruction, segmentation etc., can be applied in an effort to extract meaning to video surveillance applications. According to the literature, these methods have been well researched and their use will be assessed in the context of current surveillance requirements in chapter 2. While existing techniques can perform well in some contexts, such as an architectural environment composed of simple geometrical elements, their robustness and performance in feature extraction and object recognition tasks is not sufficient to solve the key challenges encountered in general video surveillance context. This is largely due to issues such as variable lighting, weather conditions, and shadows and in general complexity of the real-world environment. Chapter 3 presents the research and contribution on those topics – methods to extract optimal features for a specific CCTV application – as well as their strengths and weaknesses to highlight that the proposed algorithm obtains better results than most due to its specific design.The comparison of current surveillance systems and methods from the literature has shown that 2D data are however almost constantly used for many applications. Indeed, industrial systems as well as the research community have been improving intensively 2D feature extraction methods since image analysis and Scene understanding has been of interest. The constant progress on 2D feature extraction methods throughout the years makes it almost effortless nowadays due to a large variety of techniques. Moreover, even if 2D data do not allow solving all challenges in video surveillance or other applications, they are still used as starting stages towards scene understanding and image analysis. Chapter 4 will then explore 2D feature extraction via vanishing point detection and segmentation methods. A combination of most common techniques and a novel approach will be then proposed to extract vanishing points from video surveillance environments. Moreover, segmentation techniques will be explored in the aim to determine how they can be used to complement vanishing point detection and lead towards 3D data extraction and analysis. In spite of the contribution above, 2D data is insufficient for all but the simplest applications aimed at obtaining an understanding of a scene, where the aim is for a robust detection of, say, left luggage or abnormal behaviour; without significant a priori information about the scene geometry. Therefore, more information is required in order to be able to design a more automated and intelligent algorithm to obtain richer information from the scene geometry and so a better understanding of what is happening within. This can be overcome by the use of 3D data (in addition to 2D data) allowing opportunity for object “classification” and from this to infer a map of functionality, describing feasible and unfeasible object functionality in a given environment. Chapter 5 presents how 3D data can be beneficial for this task and the various solutions investigated to recover 3D data, as well as some preliminary work towards plane extraction.It is apparent that VPs and planes give useful information about a scene’s perspective and can assist in 3D data recovery within a scene. However, neither VPs nor plane detection techniques alone allow the recovery of more complex generic object shapes - for example composed of spheres, cylinders etc - and any simple model will suffer in the presence of non-Manhattan features, e.g. introduced by the presence of an escalator. For this reason, a novel photometric stereo-based surface normal retrieval methodology is introduced to capture the 3D geometry of the whole scene or part of it. Chapter 6 describes how photometric stereo allows recovery of 3D information in order to obtain a better understanding of a scene, as well as also partially overcoming some current surveillance challenges, such as difficulty in resolving fine detail, particularly at large standoff distances, and in isolating and recognising more complex objects in real scenes. Here items of interest may be obscured by complex environmental factors that are subject to rapid change, making, for example, the detection of suspicious objects and behaviour highly problematic. Here innovative use is made of an untapped latent capability offered within modern surveillance environments to introduce a form of environmental structuring to good advantage in order to achieve a richer form of data acquisition. This chapter also goes on to explore the novel application of photometric stereo in such diverse applications, how our algorithm can be incorporated into an existing surveillance system and considers a typical real commercial application.One of the most important aspects of this research work is its application. Indeed, while most of the research literature has been based on relatively simple structured environments, the approach here has been designed to be applied to real surveillance environments, such as railway stations, airports, waiting rooms, etc, and where surveillance cameras may be fixed or in the future form part of a mobile robotic free roaming surveillance device, that must continually reinterpret its changing environment. So, as mentioned previously, while the main focus has been to apply this algorithm to railway station environments, the work has been approached in a way that allows adaptation to many other applications, such as autonomous robotics, and in motorway, shopping centre, street and home environments. All of these applications require a better understanding of the scene for security or safety purposes. Finally, chapter 7 presents a global conclusion and what will be achieved in the future

Road environment modeling using robust perspective analysis and recursive Bayesian segmentation

Recently, vision-based advanced driver-assistance systems (ADAS) have received a new increased interest to enhance driving safety. In particular, due to its high performance–cost ratio, mono-camera systems are arising as the main focus of this field of work. In this paper we present a novel on-board road modeling and vehicle detection system, which is a part of the result of the European I-WAY project. The system relies on a robust estimation of the perspective of the scene, which adapts to the dynamics of the vehicle and generates a stabilized rectified image of the road plane. This rectified plane is used by a recursive Bayesian classi- fier, which classifies pixels as belonging to different classes corresponding to the elements of interest of the scenario. This stage works as an intermediate layer that isolates subsequent modules since it absorbs the inherent variability of the scene. The system has been tested on-road, in different scenarios, including varied illumination and adverse weather conditions, and the results have been proved to be remarkable even for such complex scenarios