Multi-sensor data fusion techniques for RPAS detect, track and avoid

Accurate and robust tracking of objects is of growing interest amongst the computer vision scientific community. The ability of a multi-sensor system to detect and track objects, and accurately predict their future trajectory is critical in the context of mission- and safety-critical applications. Remotely Piloted Aircraft System (RPAS) are currently not equipped to routinely access all classes of airspace since certified Detect-and-Avoid (DAA) systems are yet to be developed. Such capabilities can be achieved by incorporating both cooperative and non-cooperative DAA functions, as well as providing enhanced communications, navigation and surveillance (CNS) services. DAA is highly dependent on the performance of CNS systems for Detection, Tacking and avoiding (DTA) tasks and maneuvers. In order to perform an effective detection of objects, a number of high performance, reliable and accurate avionics sensors and systems are adopted including non-cooperative sensors (visual and thermal cameras, Laser radar (LIDAR) and acoustic sensors) and cooperative systems (Automatic Dependent Surveillance-Broadcast (ADS-B) and Traffic Collision Avoidance System (TCAS)). In this paper the sensors and system information candidates are fully exploited in a Multi-Sensor Data Fusion (MSDF) architecture. An Unscented Kalman Filter (UKF) and a more advanced Particle Filter (PF) are adopted to estimate the state vector of the objects based for maneuvering and non-maneuvering DTA tasks. Furthermore, an artificial neural network is conceptualised/adopted to exploit the use of statistical learning methods, which acts to combined information obtained from the UKF and PF. After describing the MSDF architecture, the key mathematical models for data fusion are presented. Conceptual studies are carried out on visual and thermal image fusion architectures

State estimation within ied based smart grid using kalman estimates

State Estimation is a traditional and reliable technique within power distribution and control systems. It is used for building a topology of the power grid network based on state measurements and current operational state of different nodes & buses. The protection of sensors and measurement units such as Intelligent Electronic Devices (IED) in Central Energy Management System (CEMS) against False Data Injection Attacks (FDIAs) is a big concern to grid operators. These are special kind of cyber-attacks that are directed towards the state & measurement data in such a way that mislead the CEMS into making incorrect decisions and create generation load imbalance. These are known to bypass the traditional bad data detection systems within central estimators. This paper presents the use of an additional novel state estimator based on Kalman filter along with traditional Distributed State Estimation (DSE) which is based on Weighted Least Square (WLS). Kalman filter is a feedback control mechanism that constantly updates itself based on state prediction and state correction technique and shows improvement in the estimates. The additional estimator output is compared with the results of DSE in order to identify anomalies and injection of false data. We evaluated our methodology by simulating proposed technique using MATPOWER over IEEE-14, IEEE-30, IEEE-118, IEEE-300 bus. The results clearly demonstrate the superiority of the proposed method over traditional state estimation. © 2021 by the authors. Licensee MDPI, Basel, Switzerland

An improved multiple model particle filtering approach for manoeuvring target tracking using airborne GMTI with geographic information

This paper proposes a ground vehicle tracking method using an airborne ground moving target indicator radar where the surrounding geographic information is considered to determine vehicle's movement type as well as constrain its positions. Multiple state models corresponding to different movement modes are applied to represent the vehicle's behaviour in different terrain conditions. Based on geographic conditions and multiple state models, a constrained variable structure multiple model particle filter algorithm is proposed. Compared with the traditional multiple model particle filtering schemes, the proposed algorithm utilises a particle swarm optimisation technique which generates more effective particles and generated particles are constrained into the feasible geographic region. Numerical simulation results in a realistic environment show that the proposed method achieves better tracking performance compared with current state-of-the-art ones for manoeuvring vehicle tracking

Architectures for embedded multimodal sensor data fusion systems in the robotics : and airport traffic suveillance ; domain

Smaller autonomous robots and embedded sensor data fusion systems often suffer from limited computational and hardware resources. Many ‘Real Time’ algorithms for multi modal sensor data fusion cannot be executed on such systems, at least not in real time and sometimes not at all, because of the computational and energy resources needed, resulting from the architecture of the computational hardware used in these systems. Alternative hardware architectures for generic tracking algorithms could provide a solution to overcome some of these limitations. For tracking and self localization sequential Bayesian filters, in particular particle filters, have been shown to be able to handle a range of tracking problems that could not be solved with other algorithms. But particle filters have some serious disadvantages when executed on serial computational architectures used in most systems. The potential increase in performance for particle filters is huge as many of the computational steps can be done concurrently. A generic hardware solution for particle filters can relieve the central processing unit from the computational load associated with the tracking task. The general topic of this research are hardware-software architectures for multi modal sensor data fusion in embedded systems in particular tracking, with the goal to develop a high performance computational architecture for embedded applications in robotics and airport traffic surveillance domain. The primary concern of the research is therefore: The integration of domain specific concept support into hardware architectures for low level multi modal sensor data fusion, in particular embedded systems for tracking with Bayesian filters; and a distributed hardware-software tracking systems for airport traffic surveillance and control systems. Runway Incursions are occurrences at an aerodrome involving the incorrect presence of an aircraft, vehicle, or person on the protected area of a surface designated for the landing and take-off of aircraft. The growing traffic volume kept runway incursions on the NTSB’s ‘Most Wanted’ list for safety improvements for over a decade. Recent incidents show that problem is still existent. Technological responses that have been deployed in significant numbers are ASDE-X and A-SMGCS. Although these technical responses are a significant improvement and reduce the frequency of runway incursions, some runway incursion scenarios are not optimally covered by these systems, detection of runway incursion events is not as fast as desired, and they are too expensive for all but the biggest airports. Local, short range sensors could be a solution to provide the necessary affordable surveillance accuracy for runway incursion prevention. In this context the following objectives shall be reached. 1) Show the feasibility of runway incursion prevention systems based on localized surveillance. 2) Develop a design for a local runway incursion alerting system. 3) Realize a prototype of the system design using the developed tracking hardware.Kleinere autonome Roboter und eingebettete Sensordatenfusionssysteme haben oft mit stark begrenzter Rechenkapazität und eingeschränkten Hardwareressourcen zu kämpfen. Viele Echtzeitalgorithmen für die Fusion von multimodalen Sensordaten können, bedingt durch den hohen Bedarf an Rechenkapazität und Energie, auf solchen Systemen überhaupt nicht ausgeführt werden, oder zu mindesten nicht in Echtzeit. Der hohe Bedarf an Energie und Rechenkapazität hat seine Ursache darin, dass die Architektur der ausführenden Hardware und der ausgeführte Algorithmus nicht aufeinander abgestimmt sind. Dies betrifft auch Algorithmen zu Spurverfolgung. Mit Hilfe von alternativen Hardwarearchitekturen für die generische Ausführung solcher Algorithmen könnten sich einige der typischerweise vorliegenden Einschränkungen überwinden lassen. Eine Reihe von Aufgaben, die sich mit anderen Spurverfolgungsalgorithmen nicht lösen lassen, lassen sich mit dem Teilchenfilter, einem Algorithmus aus der Familie der Bayesschen Filter lösen. Bei der Ausführung auf traditionellen Architekturen haben Teilchenfilter gegenüber anderen Algorithmen einen signifikanten Nachteil, allerdings ist hier ein großer Leistungszuwachs durch die nebenläufige Ausführung vieler Rechenschritte möglich. Eine generische Hardwarearchitektur für Teilchenfilter könnte deshalb die oben genannten Systeme stark entlasten. Das allgemeine Thema dieses Forschungsvorhabens sind Hardware-Software-Architekturen für die multimodale Sensordatenfusion auf eingebetteten Systemen - speziell für Aufgaben der Spurverfolgung, mit dem Ziel eine leistungsfähige Architektur für die Berechnung entsprechender Algorithmen auf eingebetteten Systemen zu entwickeln, die für Anwendungen in der Robotik und Verkehrsüberwachung auf Flughäfen geeignet ist. Das Augenmerk des Forschungsvorhabens liegt dabei auf der Integration von vom Einsatzgebiet abhängigen Konzepten in die Architektur von Systemen zur Spurverfolgung mit Bayeschen Filtern, sowie auf verteilten Hardware-Software Spurverfolgungssystemen zur Überwachung und Führung des Rollverkehrs auf Flughäfen. Eine „Runway Incursion“ (RI) ist ein Vorfall auf einem Flugplatz, bei dem ein Fahrzeug oder eine Person sich unerlaubt in einem Abschnitt der Start- bzw. Landebahn befindet, der einem Verkehrsteilnehmer zur Benutzung zugewiesen wurde. Der wachsende Flugverkehr hat dafür gesorgt, das RIs seit über einem Jahrzehnt auf der „Most Wanted“-Liste des NTSB für Verbesserungen der Sicherheit stehen. Jüngere Vorfälle zeigen, dass das Problem noch nicht behoben ist. Technologische Maßnahmen die in nennenswerter Zahl eingesetzt wurden sind das ASDE-X und das A-SMGCS. Obwohl diese Maßnahmen eine deutliche Verbesserung darstellen und die Zahl der RIs deutlich reduzieren, gibt es einige RISituationen die von diesen Systemen nicht optimal abgedeckt werden. Außerdem detektieren sie RIs ist nicht so schnell wie erwünscht und sind - außer für die größten Flughäfen - zu teuer. Lokale Sensoren mit kurzer Reichweite könnten eine Lösung sein um die für die zuverlässige Erkennung von RIs notwendige Präzision bei der Überwachung des Rollverkehrs zu erreichen. Vor diesem Hintergrund sollen die folgenden Ziele erreicht werden. 1) Die Machbarkeit eines Runway Incursion Vermeidungssystems, das auf lokalen Sensoren basiert, zeigen. 2) Einen umsetzbaren Entwurf für ein solches System entwickeln. 3) Einen Prototypen des Systems realisieren, das die oben gennannte Hardware zur Spurverfolgung einsetzt

False data injection attack detection in smart grid

Smart grid is a distributed and autonomous energy delivery infrastructure that constantly monitors the operational state of its overall network using smart techniques and state estimation. State estimation is a powerful technique that is used to determine the overall operational state of the system based on a limited set of measurements collected through metering systems. Cyber-attacks pose serious risks to a smart grid state estimation that can cause disruptions and power outages resulting in huge economical losses and are therefore a big concern to a reliable national grid operation. False data injection attacks (FDIAs), engineered on the basis of the knowledge of the network configuration, are difficult to detect using the traditional data detection mechanisms. These detection schemes have been found vulnerable and failed to detect these FDIAs. FDIAs specifically target the state data and can manipulate the state measurements in such a way that these false measurements appear real to the main control systems. This research work explores the possibility of FDIA detection using state estimation in a distributed and partitioned smart grid. In order to detect FDIAs we use measurements for residual-based testing which creates an objective function; and the probability of erroneous data is determined from this residual test. In this test, a preset threshold is determined based on the prior history of the state data. FDIA cases are simulated within a smart grid considering that the Chi-square detection state estimator fails in identifying such attacks. We compute the objective function using the standard weighted least problem and then test the objective function against the value in the Chi-square table. The gain matrix and the Jacobian matrix are computed. The state variables are computed in the form of a voltage magnitude. The state variables are computed after the inception of an attack to assess these state magnitude results. Different sizes of partitioning are used to improve the overall sensitivity of the Chi-square results. Our additional estimator is based on a Kalman estimation that consists of the state prediction and state correction steps. In the first step, it obtains the state and matrix covariance prediction, and in the second step, it calculates the Kalman gain and the state and matrix covariance update steps. The set of points is created for the state vector x at a time instant t. The initial vector and covariance matrix are based on a priori knowledge of the historical estimates. A set of sigma points is estimated by the state update function. Sigma points refer to the minimal set of sampling points that are selected and transformed using nonlinear function, and the new mean and the covariance are formed out of these transformed points. The idea behind this is that it is easier to compute a Gaussian distribution than an arbitrary nonlinear function. The filter gain, the mean and the covariance are used to estimate the next state. Our simulation results show that the combination of Kalman estimation and distributed state estimation improves the overall stability index and vulnerability assessment score of the smart grid. We built a stability index table for a smart grid based on the state estimates value after the inception of an FDIA. The vulnerability assessment score of the smart grid is based on common vulnerability scoring system (CVSS) and state estimates under the influence of an FDIA. The simulations are conducted in the MATPOWER program and different electrical bus systems such as IEEE 14, 30, 39, 118 and 300 are tested. All the contributions have been published in reputable journals and conferences.Doctor of Philosoph

A Fault-Tolerant Multiple Sensor Fusion Approach Applied to UAV Attitude Estimation

A novel sensor fusion design framework is presented with the objective of improving the overall multisensor measurement system performance and achieving graceful degradation following individual sensor failures. The Unscented Information Filter (UIF) is used to provide a useful tool for combining information from multiple sources. A two-step off-line and on-line calibration procedure refines sensor error models and improves the measurement performance. A Fault Detection and Identification (FDI) scheme crosschecks sensor measurements and simultaneously monitors sensor biases. Low-quality or faulty sensor readings are then rejected from the final sensor fusion process. The attitude estimation problem is used as a case study for the multiple sensor fusion algorithm design, with information provided by a set of low-cost rate gyroscopes, accelerometers, magnetometers, and a single-frequency GPS receiver’s position and velocity solution. Flight data collected with an Unmanned Aerial Vehicle (UAV) research test bed verifies the sensor fusion, adaptation, and fault-tolerance capabilities of the designed sensor fusion algorithm