Real-Time, Multiple Pan/Tilt/Zoom Computer Vision Tracking and 3D Positioning System for Unmanned Aerial System Metrology

The study of structural characteristics of Unmanned Aerial Systems (UASs) continues to be an important field of research for developing state of the art nano/micro systems. Development of a metrology system using computer vision (CV) tracking and 3D point extraction would provide an avenue for making these theoretical developments. This work provides a portable, scalable system capable of real-time tracking, zooming, and 3D position estimation of a UAS using multiple cameras. Current state-of-the-art photogrammetry systems use retro-reflective markers or single point lasers to obtain object poses and/or positions over time. Using a CV pan/tilt/zoom (PTZ) system has the potential to circumvent their limitations. The system developed in this paper exploits parallel-processing and the GPU for CV-tracking, using optical flow and known camera motion, in order to capture a moving object using two PTU cameras. The parallel-processing technique developed in this work is versatile, allowing the ability to test other CV methods with a PTZ system using known camera motion. Utilizing known camera poses, the object\u27s 3D position is estimated and focal lengths are estimated for filling the image to a desired amount. This system is tested against truth data obtained using an industrial system

Intelligent Municipal Heritage Management Service in a Smart City: Telecommunication Traffic Characterizationand Quality of Service

[EN] The monitoring of cultural heritage is becoming common in cities to provide heritage preservation and prevent vandalism. Using sensors and video cameras for this task implies the need to transmit information. In this paper, the teletraffic that cameras and sensors generate is characterized and the transmissions¿ influence on the municipal communications network is evaluated. Then, we propose models for telecommunication traffic sources in an intelligent municipal heritage management service inside a smart sustainable city. The sources were simulated in a smart city scenario to find the proper quality of service (QoS) parameters for the communication network, using Valencia City as background. Specific sensors for intelligent municipal heritage management were selected and four telecommunication traffic sources were modelled according to real-life requirements and sensors datasheet. Different simulations were performed to find the proper CIR (Committed Information Rate) and PIR (Peak Information Rate) values and to study the effects of limited bandwidth networks. Packet loss, throughput, delay, and jitter were used to evaluate the network¿s performance. Consequently, the result was the selection of the minimum values for PIR and CIR that ensured QoS and thus optimized the traffic telecommunication costs associated with an intelligent municipal heritage management service.This work was partially supported by Spanish Government Projects TIN2013-47272-C2-1-R and TEC2015-71932-REDTRodríguez-Hernández, MA.; Jiang, Z.; Gomez-Sacristan, Á.; Pla, V. (2019). Intelligent Municipal Heritage Management Service in a Smart City: Telecommunication Traffic Characterizationand Quality of Service. Wireless Communications and Mobile Computing (Online). 1-10. https://doi.org/10.1155/2019/8412542S11

Development of artificial neural network-based object detection algorithms for low-cost hardware devices

Finally, the fourth work was published in the “WCCI” conference in 2020 and consisted of an individuals' position estimation algorithm based on a novel neural network model for environments with forbidden regions, named “Forbidden Regions Growing Neural Gas”.The human brain is the most complex, powerful and versatile learning machine ever known. Consequently, many scientists of various disciplines are fascinated by its structures and information processing methods. Due to the quality and quantity of the information extracted from the sense of sight, image is one of the main information channels used by humans. However, the massive amount of video footage generated nowadays makes it difficult to process those data fast enough manually. Thus, computer vision systems represent a fundamental tool in the extraction of information from digital images, as well as a major challenge for scientists and engineers. This thesis' primary objective is automatic foreground object detection and classification through digital image analysis, using artificial neural network-based techniques, specifically designed and optimised to be deployed in low-cost hardware devices. This objective will be complemented by developing individuals' movement estimation methods by using unsupervised learning and artificial neural network-based models. The cited objectives have been addressed through a research work illustrated in four publications supporting this thesis. The first one was published in the “ICAE” journal in 2018 and consists of a neural network-based movement detection system for Pan-Tilt-Zoom (PTZ) cameras deployed in a Raspberry Pi board. The second one was published in the “WCCI” conference in 2018 and consists of a deep learning-based automatic video surveillance system for PTZ cameras deployed in low-cost hardware. The third one was published in the “ICAE” journal in 2020 and consists of an anomalous foreground object detection and classification system for panoramic cameras, based on deep learning and supported by low-cost hardware

POINTING, ACQUISITION, AND TRACKING FOR DIRECTIONAL WIRELESS COMMUNICATIONS NETWORKS

Directional wireless communications networks (DWNs) are expected to become a workhorse of the military, as they provide great network capacity in hostile areas where omnidirectional RF systems can put their users in harm's way. These networks will also be able to adapt to new missions, change topologies, use different communications technologies, yet still reliably serve all their terminal users. DWNs also have the potential to greatly expand the capacity of civilian and commercial wireless communication. The inherently narrow beams present in these types of systems require a means of steering them, acquiring the links, and tracking to maintain connectivity. This area of technological challenges encompasses all the issues of pointing, acquisition, and tracking (PAT). iii The two main technologies for DWNs are Free-Space Optical (FSO) and millimeter wave RF (mmW). FSO offers tremendous bandwidths, long ranges, and uses existing fiber-based technologies. However, it suffers from severe turbulence effects when passing through long (>kms) atmospheric paths, and can be severely affected by obscuration. MmW systems do not suffer from atmospheric effects nearly as much, use much more sensitive coherent receivers, and have wider beam divergences allowing for easier pointing. They do, however, suffer from a lack of available small-sized power amplifiers, complicated RF infrastructure that must be steered with a platform, and the requirement that all acquisition and tracking be done with the data beam, as opposed to FSO which uses a beacon laser for acquisition and a fast steering mirror for tracking. This thesis analyzes the many considerations required for designing and implementing a FSO PAT system, and extends this work to the rapidly expanding area of mmW DWN systems. Different types of beam acquisition methods are simulated and tested, and the tradeoffs between various design specifications are analyzed and simulated to give insight into how to best implement a transceiver platform. An experimental test-bed of six FSO platforms is also designed and constructed to test some of these concepts, along with the implementation of a three-node biconnected network. Finally, experiments have been conducted to assess the performance of fixed infrastructure routing hardware when operating with a physically reconfigurable RF network

Low cost network camera sensors for traffic monitoring

Report on a study investigating the ways new video and wireless technology can be implemented into Texas Department of Transportation video monitoring systems to increase efficiency and reduce costs

Wide-Area Surveillance System using a UAV Helicopter Interceptor and Sensor Placement Planning Techniques

This project proposes and describes the implementation of a wide-area surveillance system comprised of a sensor/interceptor placement planning and an interceptor unmanned aerial vehicle (UAV) helicopter. Given the 2-D layout of an area, the planning system optimally places perimeter cameras based on maximum coverage and minimal cost. Part of this planning system includes the MATLAB implementation of Erdem and Sclaroff’s Radial Sweep algorithm for visibility polygon generation. Additionally, 2-D camera modeling is proposed for both fixed and PTZ cases. Finally, the interceptor is also placed to minimize shortest-path flight time to any point on the perimeter during a detection event. Secondly, a basic flight control system for the UAV helicopter is designed and implemented. The flight control system’s primary goal is to hover the helicopter in place when a human operator holds an automatic-flight switch. This system represents the first step in a complete waypoint-navigation flight control system. The flight control system is based on an inertial measurement unit (IMU) and a proportional-integral-derivative (PID) controller. This system is implemented using a general-purpose personal computer (GPPC) running Windows XP and other commercial off-the-shelf (COTS) hardware. This setup differs from other helicopter control systems which typically use custom embedded solutions or micro-controllers. Experiments demonstrate the sensor placement planning achieving \u3e90% coverage at optimized-cost for several typical areas given multiple camera types and parameters. Furthermore, the helicopter flight control system experiments achieve hovering success over short flight periods. However, the final conclusion is that the COTS IMU is insufficient for high-speed, high-frequency applications such as a helicopter control system

Mobile robot vavigation using a vision based approach

PhD ThesisThis study addresses the issue of vision based mobile robot navigation in a partially cluttered indoor environment using a mapless navigation strategy. The work focuses on two key problems, namely vision based obstacle avoidance and vision based reactive navigation strategy. The estimation of optical flow plays a key role in vision based obstacle avoidance problems, however the current view is that this technique is too sensitive to noise and distortion under real conditions. Accordingly, practical applications in real time robotics remain scarce. This dissertation presents a novel methodology for vision based obstacle avoidance, using a hybrid architecture. This integrates an appearance-based obstacle detection method into an optical flow architecture based upon a behavioural control strategy that includes a new arbitration module. This enhances the overall performance of conventional optical flow based navigation systems, enabling a robot to successfully move around without experiencing collisions. Behaviour based approaches have become the dominant methodologies for designing control strategies for robot navigation. Two different behaviour based navigation architectures have been proposed for the second problem, using monocular vision as the primary sensor and equipped with a 2-D range finder. Both utilize an accelerated version of the Scale Invariant Feature Transform (SIFT) algorithm. The first architecture employs a qualitative-based control algorithm to steer the robot towards a goal whilst avoiding obstacles, whereas the second employs an intelligent control framework. This allows the components of soft computing to be integrated into the proposed SIFT-based navigation architecture, conserving the same set of behaviours and system structure of the previously defined architecture. The intelligent framework incorporates a novel distance estimation technique using the scale parameters obtained from the SIFT algorithm. The technique employs scale parameters and a corresponding zooming factor as inputs to train a neural network which results in the determination of physical distance. Furthermore a fuzzy controller is designed and integrated into this framework so as to estimate linear velocity, and a neural network based solution is adopted to estimate the steering direction of the robot. As a result, this intelligent iv approach allows the robot to successfully complete its task in a smooth and robust manner without experiencing collision. MS Robotics Studio software was used to simulate the systems, and a modified Pioneer 3-DX mobile robot was used for real-time implementation. Several realistic scenarios were developed and comprehensive experiments conducted to evaluate the performance of the proposed navigation systems. KEY WORDS: Mobile robot navigation using vision, Mapless navigation, Mobile robot architecture, Distance estimation, Vision for obstacle avoidance, Scale Invariant Feature Transforms, Intelligent framework

Requirements for digitized aircraft spotting (Ouija) board for use on U.S. Navy aircraft carriers

This thesis will evaluate system and process elements to initiate requirements modeling necessary for the next generation Digitized Aircraft Spotting (Ouija) Board for use on U.S. Navy aircraft carriers to track and plan aircraft movement. The research will examine and evaluate the feasibility and suitability of transforming the existing two-dimensional static board to an electronic, dynamic display that will enhance situational awareness by using sensors and system information from various sources to display a comprehensive operational picture of the current flight and hangar decks aboard aircraft carriers. The authors will evaluate the current processes and make recommendations on elements the new system would display. These elements include what information is displayed, which external systems feed information to the display, and how intelligent agents could be used to transform the static display to a powerful decision support tool. Optimally, the Aircraft Handler will use this system to effectively manage the Flight and Hangar decks to support the projection of air power from U.S. aircraft carriers.http://archive.org/details/requirementsford109454447Lieutenant Commander, United States NavyLieutenant Commander, United States Navy ReserveApproved for public release; distribution is unlimited