USING REINFORCEMENT LEARNING TO SPOOF A MONITORED KALMAN FILTER

Modern hardware systems rely on state estimators such as Kalman filters to monitor key variables for feedback and performance monitoring. The performance of the hardware system can be monitored using a chi-squared fault detection test. Previous work has shown that Kalman filters are susceptible to false data injection attacks. In a false data injection attack, intentional noise and/or bias is added to sensor measurement data to mislead a Kalman filter in a way that goes undetected by the chi-squared test. This thesis proposes a method to deceive a Kalman filter where the attack data is generated using reinforcement learning. It is shown that reinforcement learning can be used to train an agent to manipulate the output of a Kalman filter via false data injection and without being detected by the chi-squared test. This result shows that machine learning can be used to successfully perform a cyber-physical attack by an actor who does not need to have in-depth knowledge and understanding of mathematics governing the operation of the target system. This result has significant real-world impact as modern smart power grids, aircraft, car, and spacecraft control systems are all cyber-physical systems that rely on trustworthy sensor data to function safely and reliably. A machine learning derived false data injection attack against any of these systems could lead to an undetected and potentially catastrophic failure.DoD SpaceLieutenant, United States NavyApproved for public release. Distribution is unlimited

Modèles algorithmes et méthodologie pour la conception de systèmes de sécurité physique basés sur des microcontrôleurs protégés des attaques cyber-physiques

Un moyen d'assurer la sécurité de systèmes basés sur des micro-contrôleurs (mCS) est de considérer une approche de génération à partir de spécifications. Malheureusement, les approches existantes souffrent d'inconvénients, et le but de la méthodologie présentée dans cette thèse est de les éviter dans le cas particulier des mCS pour la sécurité physique (mCS-Sec). Les principaux résultats de ce travail sont le développement de modèles, d'algorithmes, et d'une méthodologie originale de création de mCS-Sec, et leur implémentation. L'applicabilité de la méthode a été évaluée sur un système de robot de surveillance d'une zone. Dans ce cas, notre évaluation a montré que l'approche développée satisfaisait toutes les contraintes imposées, tout en offrant certains avantages par rapport aux solutions existantes. Nous pensons que cette approche permettra de réduire le nombre de faiblesses et les problèmes d'architecture dans les mCS, ce qui en réduira la surface d'attaque.One of the possible ways to ensure the security of microcontroller-based systems is the implementation of security by design approach. Unfortunately, existing approaches are not without drawbacks, that is why this thesis is aimed at developing the new one. Moreover, among all possible systems, in this work, only physical security systems were chosen as an area of the application. The main findings of the work are containing original models, algorithms, methodology and software implementation. Their correctness was checked on a system of mobile robots for perimeter monitoring. The evaluation of the developed solution showed that it satisfies all requirements while having advantages over commercial and scientific analogues, which means that the goal of this work was reached. It is assumed that the use of the developed solution will help to reduce the number of weak places and architectural defects in microcontroller-based systems, thereby significantly reducing their attack surface