Stabilized RPA Flight in Building Proximity Operations

The thesis seeks a solution to the requirement for a highly reliable and capable Unmanned Air Vehicle (UAV) to support a wide array of missions and applications that require close proximity flight to structures. The scope of the project includes the drafting of a concept of operations (CONOPs) describing how the mission requirements might be met using the sensor, operators, and air vehicle described here in. The demonstration of the wall-following section of that CONOPs is performed by cart testing a custom algorithm and evaluating its ability to react to its environment. Finally, a flight test was performed to characterize the capabilities of an RTK-GPS system to stably hold a UAV in a single position, and minimize vehicle yaw, as a potential means of minimizing environmental sensing requirements in GPS permissive environments. The results for RTK-GPS were, position hold standard of deviation 8.0 x 10.1cm at a 5m flight altitude, and 17cm x 12.7cm at 8m flight altitude. Yaw variation results were a standard of deviation of 1.7° at 5m and 3.7° at 8m. The LIDAR wall-following tests proved the feasibility of using a decision tree style coding approach to proximity flight near a structure, but still has some changes that should be considered before being used operationally

Using Helmet Mounted Displays to Designate and Locate Targets in the Urban Environment

Technologies have developed within the last ten years to allow the Helmet Mounted Display (HMD) to be much more effective as an air-to-ground (A/G) weapons cue. HMD A/G accuracy and performance requirements should be added to the Joint Helmet Mounted Cueing System (JHMCS) specifications, detailed to be as good or better than the FA-18 heads-up-display (HUD). Because of target ranging and line-of-sight (LOS) errors, the JHMCS is only used as an area sensor cue in the urban close air support (CAS) role. Therefore, for use against point targets, improvements to JHMCS are needed. LOS errors have to be reduced from the current 13-mil error, which would equate to +/- 260 feet from a 20,000 ft slant range. To decrease this error, more accurate helmet trackers must be used with faster update rates. HMD Earth referenced symbol update rates, which are currently restricted to 20 Hz, must be increased to allow the helmet to provide accurate information, despite aggressive maneuvering or operations in a turbulent environment. Accurate ranging sources must be developed to enhance the target elevation algorithm in the FA-18 to ensure usable target data, once designations are made. During turbulent flight conditions, the difference between the actual target position on the ground and the unstable target designation (TD) diamond depicting it cause motion differences, which distract the pilot. Methods to filter the movement of earth-referenced symbols should be explored, as well as increasing JHMCS symbol write rates. Additionally, vibration levels during low-level flight and moderate turbulence levels make HMD A/G aiming and designation tasks very difficult. Buffet suppression algorithms are used during vibrations in the air-to-air (A/A) aiming role and should be implemented for A/G use as well. The purpose of this study is to focus on present capabilities with JHMCS. The author’s tactical experience has been achieved on the FA-18 A-F variants and tactical applicability will be directed to that platform. While most references to helmet displays will center on lessons learned from the JHMCS, helmet mounted display experience was gained while serving as an exchange officer with the UK Royal Air Force and evaluating the Guardian HMD system. The analysis contained within this thesis is based on the operational insights of operating within the demanding Close Air Support (CAS) environment and the tactical enhancement that has been demonstrated with the use of Helmet Mounted Cueing systems. Currently, JHMCS is available to about half the FA-18 fleet and operational assessments, resulting from its use in the Iraqi conflict, has accelerated the demand for increased capabilities to this target cueing device. Lessons learned from the current generation of HMDs will play a major role in the design of the cockpit for the Joint Strike Fighter (JSF)

Combining visual, pedestrian, and collaborative navigation techniques for team based infrastructure free indoor navigation

Peer reviewe

Fireground location understanding by semantic linking of visual objects and building information models

This paper presents an outline for improved localization and situational awareness in fire emergency situations based on semantic technology and computer vision techniques. The novelty of our methodology lies in the semantic linking of video object recognition results from visual and thermal cameras with Building Information Models (BIM). The current limitations and possibilities of certain building information streams in the context of fire safety or fire incident management are addressed in this paper. Furthermore, our data management tools match higher-level semantic metadata descriptors of BIM and deep-learning based visual object recognition and classification networks. Based on these matches, estimations can be generated of camera, objects and event positions in the BIM model, transforming it from a static source of information into a rich, dynamic data provider. Previous work has already investigated the possibilities to link BIM and low-cost point sensors for fireground understanding, but these approaches did not take into account the benefits of video analysis and recent developments in semantics and feature learning research. Finally, the strengths of the proposed approach compared to the state-of-the-art is its (semi -)automatic workflow, generic and modular setup and multi-modal strategy, which allows to automatically create situational awareness, to improve localization and to facilitate the overall fire understanding

Error Modelling for Multi-Sensor Measurements in Infrastructure-Free Indoor Navigation

The long-term objective of our research is to develop a method for infrastructure-free simultaneous localization and mapping (SLAM) and context recognition for tactical situational awareness. Localization will be realized by propagating motion measurements obtained using a monocular camera, a foot-mounted Inertial Measurement Unit (IMU), sonar, and a barometer. Due to the size and weight requirements set by tactical applications, Micro-Electro-Mechanical (MEMS) sensors will be used. However, MEMS sensors suffer from biases and drift errors that may substantially decrease the position accuracy. Therefore, sophisticated error modelling and implementation of integration algorithms are key for providing a viable result. Algorithms used for multi-sensor fusion have traditionally been different versions of Kalman filters. However, Kalman filters are based on the assumptions that the state propagation and measurement models are linear with additive Gaussian noise. Neither of the assumptions is correct for tactical applications, especially for dismounted soldiers, or rescue personnel. Therefore, error modelling and implementation of advanced fusion algorithms are essential for providing a viable result. Our approach is to use particle filtering (PF), which is a sophisticated option for integrating measurements emerging from pedestrian motion having non-Gaussian error characteristics. This paper discusses the statistical modelling of the measurement errors from inertial sensors and vision based heading and translation measurements to include the correct error probability density functions (pdf) in the particle filter implementation. Then, model fitting is used to verify the pdfs of the measurement errors. Based on the deduced error models of the measurements, particle filtering method is developed to fuse all this information, where the weights of each particle are computed based on the specific models derived. The performance of the developed method is tested via two experiments, one at a university’s premises and another in realistic tactical conditions. The results show significant improvement on the horizontal localization when the measurement errors are carefully modelled and their inclusion into the particle filtering implementation correctly realized

Automated Find Fix and Track with a Medium Altitude Long Endurance Remotely Piloted Aircraft

A limitation in RPA ISR operations is loss of target track if the command link is severed. For an RPA to effectively execute the ISR mission without a command link, it needs the capability to F2T targets autonomously. Automated Find Fix and Track (AFFTRAC) was developed to help solve this problem by demonstrating a proof of concept tactical autopilot. Monocular stereo vision was used to process sequential images acquired during orbit to produce a partial structural point cloud of the original structure. This partial structural point cloud was then exploited to create a holding area density for the aircraft to stay within. A simple greedy algorithm exploited this holding area density to produce aircraft turn commands to approximate tactical ISR holding. The result was that imagery from existing MQ-9 sensors was used to provide command guidance to autonomously to maintain line of sight to a target. Overall, AFFTRAC is a promising initial framework for a tactical autopilot, but additional development is needed to mature component algorithms

A Decision Analysis Framework for Evaluation of Helmet Mounted Display Alternatives for Fighter Aircraft

The promise of providing an intuitive and efficient information interface, while allowing the warfighter to perform other critical tasks such as targeting or aircraft control, has led to the growing popularity of Helmet Mounted Displays (HMDs) across the military landscape, especially combat aircraft. Though design and selection of competing systems is critical to optimized performance and safety, structured methods for the evaluation of HMDs are not often used in the acquisition community, leaving selection among alternative designs to the judgment of subject matter experts. However, technical decision-making has been shown to be flawed without the use of a structured decision analysis framework, which can help to overcome narrow focus, potential bias, and human error. This thesis proposes a HMD design evaluation framework that derives system metrics from fundamental multi-level performance objectives and employs a robust, analytical approach to assess the alternative\u27s ability to bring value to these objectives. Supported by principles of Human Systems Integration (HSI) and Value-Focused Thinking, the framework can be used by decision makers to craft informed, defendable judgments that strive to increase system performance while decreasing maintenance and integration resource. The 17-factor framework is illustrated through application on two possible solutions for a fixed-wing fighter platform

A Monocular SLAM Method to Estimate Relative Pose During Satellite Proximity Operations

Automated satellite proximity operations is an increasingly relevant area of mission operations for the US Air Force with potential to significantly enhance space situational awareness (SSA). Simultaneous localization and mapping (SLAM) is a computer vision method of constructing and updating a 3D map while keeping track of the location and orientation of the imaging agent inside the map. The main objective of this research effort is to design a monocular SLAM method customized for the space environment. The method developed in this research will be implemented in an indoor proximity operations simulation laboratory. A run-time analysis is performed, showing near real-time operation. The method is verified by comparing SLAM results to truth vertical rotation data from a CubeSat air bearing testbed. This work enables control and testing of simulated proximity operations hardware in a laboratory environment. Additionally, this research lays the foundation for autonomous satellite proximity operations with unknown targets and minimal additional size, weight, and power requirements, creating opportunities for numerous mission concepts not previously available