Mixed-reality for unmanned aerial vehicle operations in near earth environments

Future applications will bring unmanned aerial vehicles (UAVs) to near Earth environments such as urban areas, causing a change in the way UAVs are currently operated. Of concern is that UAV accidents still occur at a much higher rate than the accident rate for commercial airliners. A number of these accidents can be attributed to a UAV pilot's low situation awareness (SA) due to the limitations of UAV operating interfaces. The main limitation is the physical separation between the vehicle and the pilot. This eliminates any motion and exteroceptive sensory feedback to the pilot. These limitation on top of a small eld of view from the onboard camera results in low SA, making near Earth operations di cult and dangerous. Autonomy has been proposed as a solution for near Earth tasks but state of the art arti cial intelligence still requires very structured and well de ned goals to allow safe autonomous operations. Therefore, there is a need to better train pilots to operate UAVs in near Earth environments and to augment their performance for increased safety and minimization of accidents.In this work, simulation software, motion platform technology, and UAV sensor suites were integrated to produce mixed-reality systems that address current limitations of UAV piloting interfaces. The mixed reality de nition is extended in this work to encompass not only the visual aspects but to also include a motion aspect. A training and evaluation system for UAV operations in near Earth environments was developed. Modi cations were made to ight simulator software to recreate current UAV operating modalities (internal and external). The training and evaluation system has been combined with Drexel's Sensor Integrated Systems Test Rig (SISTR) to allow simulated missions while incorporating real world environmental e ects andUAV sensor hardware.To address the lack of motion feedback to a UAV pilot, a system was developed that integrates a motion simulator into UAV operations. The system is designed such that during ight, the angular rate of a UAV is captured by an onboard inertial measurement unit (IMU) and is relayed to a pilot controlling the vehicle from inside the motion simulator.Efforts to further increase pilot SA led to the development of a mixed reality chase view piloting interface. Chase view is similar to a view of being towed behind the aircraft. It combines real world onboard camera images with a virtual representation of the vehicle and the surrounding operating environment. A series of UAV piloting experiments were performed using the training and evaluation systems described earlier. Subjects' behavioral performance while using the onboard camera view and the mixed reality chase view interface during missions was analyzed. Subjects' cognitive workload during missions was also assessed using subjective measures such as NASA task load index and non-subjective brain activity measurements using a functional Infrared Spectroscopy (fNIR) system. Behavioral analysis showed that the chase view interface improved pilot performance in near Earth ights and increased their situational awareness. fNIR analysis showed that a subjects cognitive workload was signi cantly less while using the chase view interface. Real world ight tests were conducted in a near Earth environment with buildings and obstacles to evaluate the chase view interface with real world data. The interface performed very well with real world, real time data in close range scenarios.The mixed reality approaches presented follow studies on human factors performance and cognitive loading. The resulting designs serve as test beds for studying UAV pilot performance, creating training programs, and developing tools to augment UAV operations and minimize UAV accidents during operations in near Earth environments.Ph.D., Mechanical Engineering -- Drexel University, 201

DRONE DELIVERY OF CBNRECy – DEW WEAPONS Emerging Threats of Mini-Weapons of Mass Destruction and Disruption (WMDD)

Drone Delivery of CBNRECy – DEW Weapons: Emerging Threats of Mini-Weapons of Mass Destruction and Disruption (WMDD) is our sixth textbook in a series covering the world of UASs and UUVs. Our textbook takes on a whole new purview for UAS / CUAS/ UUV (drones) – how they can be used to deploy Weapons of Mass Destruction and Deception against CBRNE and civilian targets of opportunity. We are concerned with the future use of these inexpensive devices and their availability to maleficent actors. Our work suggests that UASs in air and underwater UUVs will be the future of military and civilian terrorist operations. UAS / UUVs can deliver a huge punch for a low investment and minimize human casualties.https://newprairiepress.org/ebooks/1046/thumbnail.jp

MODELLING VIRTUAL ENVIRONMENT FOR ADVANCED NAVAL SIMULATION

This thesis proposes a new virtual simulation environment designed as element of an interoperable federation of simulator to support the investigation of complex scenarios over the Extended Maritime Framework (EMF). Extended Maritime Framework is six spaces environment (Underwater, Water surface, Ground, Air, Space, and Cyberspace) where parties involved in Joint Naval Operations act. The amount of unmanned vehicles involved in the simulation arise the importance of the Communication modelling, thus the relevance of Cyberspace. The research is applied to complex cases (one applied to deep waters and one to coast and littoral protection) as examples to validate this approach; these cases involve different kind of traditional assets (e.g. satellites, helicopters, ships, submarines, underwater sensor infrastructure, etc.) interact dynamically and collaborate with new autonomous systems (i.e. AUV, Gliders, USV and UAV). The use of virtual simulation is devoted to support validation of new concepts and investigation of collaborative engineering solutions by providing a virtual representation of the current situation; this approach support the creation of dynamic interoperable immersive framework that could support training for Man in the Loop, education and tactical decision introducing the Man on the Loop concepts. The research and development of the Autonomous Underwater Vehicles requires continuous testing so a time effective approach can result a very useful tool. In this context the simulation can be useful to better understand the behaviour of Unmanned Vehicles and to avoid useless experimentations and their costs finding problems before doing them. This research project proposes the creation of a virtual environment with the aim to see and understand a Joint Naval Scenario. The study will be focusing especially on the integration of Autonomous Systems with traditional assets; the proposed simulation deals especially with collaborative operation involving different types of Autonomous Underwater Vehicles (AUV), Unmanned Surface Vehicles (USV) and UAV (Unmanned Aerial Vehicle). The author develops an interoperable virtual simulation devoted to present the overall situation for supervision considering also the sensor capabilities, communications and mission effectiveness that results dependent of the different asset interaction over a complex heterogeneous network. The aim of this research is to develop a flexible virtual simulation solution as crucial element of an HLA federation able to address the complexity of Extended Maritime Framework (EMF). Indeed this new generation of marine interoperable simulation is a strategic advantage for investigating the problems related to the operational use of autonomous systems and to finding new ways to use them respect to different scenarios. The research deal with the creation of two scenarios, one related to military operations and another one on coastal and littoral protection where the virtual simulation propose the overall situation and allows to navigate into the virtual world considering the complex physics affecting movement, perception, interaction and communication. By this approach, it becomes evident the capability to identify, by experimental analysis within the virtual world, the new solutions in terms of engineering and technological configuration of the different systems and vehicles as well as new operational models and tactics to address the specific mission environment. The case of study is a maritime scenario with a representation of heterogeneous network frameworks that involves multiple vehicles both naval and aerial including AUVs, USVs, gliders, helicopter, ships, submarines, satellite, buoys and sensors. For the sake of clarity aerial communications will be represented divided from underwater ones. A connection point for the latter will be set on the keel line of surface vessels representing communication happening via acoustic modem. To represent limits in underwater communications, underwater signals have been considerably slowed down in order to have a more realistic comparison with aerial ones. A maximum communication distance is set, beyond which no communication can take place. To ensure interoperability the HLA Standard (IEEE 1516 evolved) is adopted to federate other simulators so to allow its extensibility for other case studies. Two different scenarios are modelled in 3D visualization: Open Water and Port Protection. The first one aims to simulate interactions between traditional assets in Extended Maritime Framework (EMF) such as satellite, navy ships, submarines, NATO Research Vessels (NRVs), helicopters, with new generation unmanned assets as AUV, Gliders, UAV, USV and the mutual advantage the subjects involved in the scenario can have; in other word, the increase in persistence, interoperability and efficacy. The second scenario models the behaviour of unmanned assets, an AUV and an USV, patrolling a harbour to find possible threats. This aims to develop an algorithm to lead patrolling path toward an optimum, guaranteeing a high probability of success in the safest way reducing human involvement in the scenario. End users of the simulation face a graphical 3D representation of the scenario where assets would be represented. He can moves in the scenario through a Free Camera in Graphic User Interface (GUI) configured to entitle users to move around the scene and observe the 3D sea scenario. In this way, players are able to move freely in the synthetic environment in order to choose the best perspective of the scene. The work is intended to provide a valid tool to evaluate the defencelessness of on-shore and offshore critical infrastructures that could includes the use of new technologies to take care of security best and preserve themselves against disasters both on economical and environmental ones

A Survey on Cellular-connected UAVs: Design Challenges, Enabling 5G/B5G Innovations, and Experimental Advancements

As an emerging field of aerial robotics, Unmanned Aerial Vehicles (UAVs) have gained significant research interest within the wireless networking research community. As soon as national legislations allow UAVs to fly autonomously, we will see swarms of UAV populating the sky of our smart cities to accomplish different missions: parcel delivery, infrastructure monitoring, event filming, surveillance, tracking, etc. The UAV ecosystem can benefit from existing 5G/B5G cellular networks, which can be exploited in different ways to enhance UAV communications. Because of the inherent characteristics of UAV pertaining to flexible mobility in 3D space, autonomous operation and intelligent placement, these smart devices cater to wide range of wireless applications and use cases. This work aims at presenting an in-depth exploration of integration synergies between 5G/B5G cellular systems and UAV technology, where the UAV is integrated as a new aerial User Equipment (UE) to existing cellular networks. In this integration, the UAVs perform the role of flying users within cellular coverage, thus they are termed as cellular-connected UAVs (a.k.a. UAV-UE, drone-UE, 5G-connected drone, or aerial user). The main focus of this work is to present an extensive study of integration challenges along with key 5G/B5G technological innovations and ongoing efforts in design prototyping and field trials corroborating cellular-connected UAVs. This study highlights recent progress updates with respect to 3GPP standardization and emphasizes socio-economic concerns that must be accounted before successful adoption of this promising technology. Various open problems paving the path to future research opportunities are also discussed.Comment: 30 pages, 18 figures, 9 tables, 102 references, journal submissio

Multi-Robot Systems: Challenges, Trends and Applications

This book is a printed edition of the Special Issue entitled “Multi-Robot Systems: Challenges, Trends, and Applications” that was published in Applied Sciences. This Special Issue collected seventeen high-quality papers that discuss the main challenges of multi-robot systems, present the trends to address these issues, and report various relevant applications. Some of the topics addressed by these papers are robot swarms, mission planning, robot teaming, machine learning, immersive technologies, search and rescue, and social robotics