Learning Realistic Traffic Agents in Closed-loop

Realistic traffic simulation is crucial for developing self-driving software in a safe and scalable manner prior to real-world deployment. Typically, imitation learning (IL) is used to learn human-like traffic agents directly from real-world observations collected offline, but without explicit specification of traffic rules, agents trained from IL alone frequently display unrealistic infractions like collisions and driving off the road. This problem is exacerbated in out-of-distribution and long-tail scenarios. On the other hand, reinforcement learning (RL) can train traffic agents to avoid infractions, but using RL alone results in unhuman-like driving behaviors. We propose Reinforcing Traffic Rules (RTR), a holistic closed-loop learning objective to match expert demonstrations under a traffic compliance constraint, which naturally gives rise to a joint IL + RL approach, obtaining the best of both worlds. Our method learns in closed-loop simulations of both nominal scenarios from real-world datasets as well as procedurally generated long-tail scenarios. Our experiments show that RTR learns more realistic and generalizable traffic simulation policies, achieving significantly better tradeoffs between human-like driving and traffic compliance in both nominal and long-tail scenarios. Moreover, when used as a data generation tool for training prediction models, our learned traffic policy leads to considerably improved downstream prediction metrics compared to baseline traffic agents. For more information, visit the project website: https://waabi.ai/rtrComment: CORL 202

Safety of autonomous vehicles: A survey on Model-based vs. AI-based approaches

The growing advancements in Autonomous Vehicles (AVs) have emphasized the critical need to prioritize the absolute safety of AV maneuvers, especially in dynamic and unpredictable environments or situations. This objective becomes even more challenging due to the uniqueness of every traffic situation/condition. To cope with all these very constrained and complex configurations, AVs must have appropriate control architectures with reliable and real-time Risk Assessment and Management Strategies (RAMS). These targeted RAMS must lead to reduce drastically the navigation risks. However, the lack of safety guarantees proves, which is one of the key challenges to be addressed, limit drastically the ambition to introduce more broadly AVs on our roads and restrict the use of AVs to very limited use cases. Therefore, the focus and the ambition of this paper is to survey research on autonomous vehicles while focusing on the important topic of safety guarantee of AVs. For this purpose, it is proposed to review research on relevant methods and concepts defining an overall control architecture for AVs, with an emphasis on the safety assessment and decision-making systems composing these architectures. Moreover, it is intended through this reviewing process to highlight researches that use either model-based methods or AI-based approaches. This is performed while emphasizing the strengths and weaknesses of each methodology and investigating the research that proposes a comprehensive multi-modal design that combines model-based and AI approaches. This paper ends with discussions on the methods used to guarantee the safety of AVs namely: safety verification techniques and the standardization/generalization of safety frameworks

Model–Based Reasoning Applied to Biological Spacecraft Payloads for Anomaly Management

With the ever-increasing complexity of spacecraft, the real-time data and state of health analysis by mission operators becomes an intricate process subject to the pitfalls of error-prone human reasoning techniques. If even detected, characterizing an anomalous state on the spacecraft can take substantial amounts of time thus reducing overall operational efficiency and possibly even jeopardizing the mission. This research specifically addresses the state-of-health analysis of biological payloads flown on NASA missions such as GeneSat-1 and PharmaSat. The complex engineering systems and timely anomaly resolution required for maintaining life support in these biological spacecraft makes human diagnosis on its own insufficient. To address these challenges, this project incorporates the use of model-based reasoning for managing anomalies found on board biological microsatellites. A spacecraft payload model was constructed with behaviors relevant to biological sample growth and the associated micro fluidic life support system. A suite of algorithms was applied to the model for computing diagnosis conjectures on detected anomalies. Implemented in MATLAB/ Simulink and designed for use as a ground-based tool for human operators, this system focused primarily on mission operator decision support. The system was developed via analysis of GeneSat-1 flight data and with a biological test bed which emulated the growth characteristics of the spacecraft. It was later integrated into the ground segment of the PharmaSat space system and verified with its flight data after launch. Results gained from experimentation validated the tool’s ability to reason on unanticipated anomalies, its speed of analysis, and its ability to augment a human operator for real-time anomaly characterization and decision support

ANALYSIS OF DOMAIN-SPECIFIC NUCLEAR ONTOLOGY USING MONTEREY PHOENIX BEHAVIOR MODELING

Current nuclear energy ontologies are known to lack a common vocabulary to formally verify nuclear energy data relationships for modeling system behaviors. Idaho National Laboratory (INL) developed the Data Integration Aggregated Model and Ontology for Nuclear Deployment (DIAMOND) ontology to provide a standard vocabulary and taxonomy for identifying data relationships in nuclear energy system models. This thesis conducted an analysis of DIAMOND using a Spent Fuel Pool (SFP) Monterey Phoenix (MP) behavior model. The SFP MP behavior modeling application demonstrated components of and interactions among a spent fuel cooling pool and its environment. The MP behavior model demonstrated a viable approach for analyzing nuclear reactor system behavior consistent with DIAMOND and the ability to generate the exhaustive set of nuclear reactor cooling pool behavior scenarios. The results supported the ability of DIAMOND definitions to be used to organize and structure knowledge about SFP’s normal and off-normal behaviors. The SPF example showed the application of assets, actions, and triggers from DIAMOND to events and relationships in MP. Assets and actions were represented as MP events, and triggers were represented as precedence relations between MP events. This thesis research verified the DIAMOND ontology was implemented correctly in the model from data representative of operationally realistic behavior and the modeling results validated the MP behavior model was well constrained.Idaho National LabCivilian, Department of the Air ForceApproved for public release. Distribution is unlimited

Data-Driven Architecture to Increase Resilience In Multi-Agent Coordinated Missions

The rise in the use of Multi-Agent Systems (MASs) in unpredictable and changing environments has created the need for intelligent algorithms to increase their autonomy, safety and performance in the event of disturbances and threats. MASs are attractive for their flexibility, which also makes them prone to threats that may result from hardware failures (actuators, sensors, onboard computer, power source) and operational abnormal conditions (weather, GPS denied location, cyber-attacks). This dissertation presents research on a bio-inspired approach for resilience augmentation in MASs in the presence of disturbances and threats such as communication link and stealthy zero-dynamics attacks. An adaptive bio-inspired architecture is developed for distributed consensus algorithms to increase fault-tolerance in a network of multiple high-order nonlinear systems under directed fixed topologies. In similarity with the natural organisms’ ability to recognize and remember specific pathogens to generate its immunity, the immunity-based architecture consists of a Distributed Model-Reference Adaptive Control (DMRAC) with an Artificial Immune System (AIS) adaptation law integrated within a consensus protocol. Feedback linearization is used to modify the high-order nonlinear model into four decoupled linear subsystems. A stability proof of the adaptation law is conducted using Lyapunov methods and Jordan decomposition. The DMRAC is proven to be stable in the presence of external time-varying bounded disturbances and the tracking error trajectories are shown to be bounded. The effectiveness of the proposed architecture is examined through numerical simulations. The proposed controller successfully ensures that consensus is achieved among all agents while the adaptive law v simultaneously rejects the disturbances in the agent and its neighbors. The architecture also includes a health management system to detect faulty agents within the global network. Further numerical simulations successfully test and show that the Global Health Monitoring (GHM) does effectively detect faults within the network

Data driven techniques for on-board performance estimation and prediction in vehicular applications.

L'abstract è presente nell'allegato / the abstract is in the attachmen

Design and implementation of machine learning techniques for modeling and managing battery energy storage systems

The fast technological evolution and industrialization that have interested the humankind since the fifties has caused a progressive and exponential increase of CO2 emissions and Earth temperature. Therefore, the research community and the political authorities have recognized the need of a deep technological revolution in both the transportation and the energy distribution systems to hinder climate changes. Thus, pure and hybrid electric powertrains, smart grids, and microgrids are key technologies for achieving the expected goals. Nevertheless, the development of the above mentioned technologies require very effective and performing Battery Energy Storage Systems (BESSs), and even more effective Battery Management Systems (BMSs). Considering the above background, this Ph.D. thesis has focused on the development of an innovative and advanced BMS that involves the use of machine learning techniques for improving the BESS effectiveness and efficiency. Great attention has been paid to the State of Charge (SoC) estimation problem, aiming at investigating solutions for achieving more accurate and reliable estimations. To this aim, the main contribution has concerned the development of accurate and flexible models of electrochemical cells. Three main modeling requirements have been pursued for ensuring accurate SoC estimations: insight on the cell physics, nonlinear approximation capability, and flexible system identification procedures. Thus, the research activity has aimed at fulfilling these requirements by developing and investigating three different modeling approaches, namely black, white, and gray box techniques. Extreme Learning Machines, Radial Basis Function Neural Networks, and Wavelet Neural Networks were considered among the black box models, but none of them were able to achieve satisfactory SoC estimation performances. The white box Equivalent Circuit Models (ECMs) have achieved better results, proving the benefit that the insight on the cell physics provides to the SoC estimation task. Nevertheless, it has appeared clear that the linearity of ECMs has reduced their effectiveness in the SoC task. Thus, the gray box Neural Networks Ensemble (NNE) and the white box Equivalent Neural Networks Circuit (ENNC) models have been developed aiming at exploiting the neural networks theory in order to achieve accurate models, ensuring at the same time very flexible system identification procedures together with nonlinear approximation capabilities. The performances of NNE and ENNC have been compelling. In particular, the white box ENNC has reached the most effective performances, achieving accurate SoC estimations, together with a simple architecture and a flexible system identification procedure. The outcome of this thesis makes it possible the development of an interesting scenario in which a suitable cloud framework provides remote assistance to several BMSs in order to adapt the managing algorithms to the aging of BESSs, even considering different and distinct applications

A model to integrate Data Mining and On-line Analytical Processing: with application to Real Time Process Control

Since the widespread use of computers in business and industry, a lot of research has been done on the design of computer systems to support the decision making task. Decision support systems support decision makers in solving unstructured decision problems by providing tools to help understand and analyze decision problems to help make better decisions. Artificial intelligence is concerned with creating computer systems that perform tasks that would require intelligence if performed by humans. Much research has focused on using artificial intelligence to develop decision support systems to provide intelligent decision support. Knowledge discovery from databases, centers around data mining algorithms to discover novel and potentially useful information contained in the large volumes of data that is ubiquitous in contemporary business organizations. Data mining deals with large volumes of data and tries to develop multiple views that the decision maker can use to study this multi-dimensional data. On-line analytical processing (OLAP) provides a mechanism that supports multiple views of multi-dimensional data to facilitate efficient analysis. These two techniques together can provide a powerful mechanism for the analysis of large quantities of data to aid the task of making decisions. This research develops a model for the real time process control of a large manufacturing process using an integrated approach of data mining and on-line analytical processing. Data mining is used to develop models of the process based on the large volumes of the process data. The purpose is to provide prediction and explanatory capability based on the models of the data and to allow for efficient generation of multiple views of the data so as to support analysis on multiple levels. Artificial neural networks provide a mechanism for predicting the behavior of nonlinear systems, while decision trees provide a mechanism for the explanation of states of systems given a set of inputs and outputs. OLAP is used to generate multidimensional views of the data and support analysis based on models developed by data mining. The architecture and implementation of the model for real-time process control based on the integration of data mining and OLAP is presented in detail. The model is validated by comparing results obtained from the integrated system, OLAP-only and expert opinion. The system is validated using actual process data and the results of this verification are presented. A discussion of the results of the validation of the integrated system and some limitations of this research with discussion on possible future research directions is provided

Dynamic learning of the environment for eco-citizen behavior

Le développement de villes intelligentes et durables nécessite le déploiement des technologies de l'information et de la communication (ITC) pour garantir de meilleurs services et informations disponibles à tout moment et partout. Comme les dispositifs IoT devenant plus puissants et moins coûteux, la mise en place d'un réseau de capteurs dans un contexte urbain peut être coûteuse. Cette thèse propose une technique pour estimer les informations environnementales manquantes dans des environnements à large échelle. Notre technique permet de fournir des informations alors que les dispositifs ne sont pas disponibles dans une zone de l'environnement non couverte par des capteurs. La contribution de notre proposition est résumée dans les points suivants : - limiter le nombre de dispositifs de détection à déployer dans un environnement urbain ; - l'exploitation de données hétérogènes acquises par des dispositifs intermittents ; - le traitement en temps réel des informations ; - l'auto-calibration du système. Notre proposition utilise l'approche AMAS (Adaptive Multi-Agent System) pour résoudre le problème de l'indisponibilité des informations. Dans cette approche, une exception est considérée comme une situation non coopérative (NCS) qui doit être résolue localement et de manière coopérative. HybridIoT exploite à la fois des informations homogènes (informations du même type) et hétérogènes (informations de différents types ou unités) acquises à partir d'un capteur disponible pour fournir des estimations précises au point de l'environnement où un capteur n'est pas disponible. La technique proposée permet d'estimer des informations environnementales précises dans des conditions de variabilité résultant du contexte d'application urbaine dans lequel le projet est situé, et qui n'ont pas été explorées par les solutions de l'état de l'art : - ouverture : les capteurs peuvent entrer ou sortir du système à tout moment sans qu'aucune configuration particulière soit nécessaire ; - large échelle : le système peut être déployé dans un contexte urbain à large échelle et assurer un fonctionnement correct avec un nombre significatif de dispositifs ; - hétérogénéité : le système traite différents types d'informations sans aucune configuration a priori. Notre proposition ne nécessite aucun paramètre d'entrée ni aucune reconfiguration. Le système peut fonctionner dans des environnements ouverts et dynamiques tels que les villes, où un grand nombre de capteurs peuvent apparaître ou disparaître à tout moment et sans aucun préavis. Nous avons fait différentes expérimentations pour comparer les résultats obtenus à plusieurs techniques standard afin d'évaluer la validité de notre proposition. Nous avons également développé un ensemble de techniques standard pour produire des résultats de base qui seront comparés à ceux obtenus par notre proposition multi-agents.The development of sustainable smart cities requires the deployment of Information and Communication Technology (ICT) to ensure better services and available information at any time and everywhere. As IoT devices become more powerful and low-cost, the implementation of an extensive sensor network for an urban context can be expensive. This thesis proposes a technique for estimating missing environmental information in large scale environments. Our technique enables providing information whereas devices are not available for an area of the environment not covered by sensing devices. The contribution of our proposal is summarized in the following points: * limiting the number of sensing devices to be deployed in an urban environment; * the exploitation of heterogeneous data acquired from intermittent devices; * real-time processing of information; * self-calibration of the system. Our proposal uses the Adaptive Multi-Agent System (AMAS) approach to solve the problem of information unavailability. In this approach, an exception is considered as a Non-Cooperative Situation (NCS) that has to be solved locally and cooperatively. HybridIoT exploits both homogeneous (information of the same type) and heterogeneous information (information of different types or units) acquired from some available sensing device to provide accurate estimates in the point of the environment where a sensing device is not available. The proposed technique enables estimating accurate environmental information under conditions of uncertainty arising from the urban application context in which the project is situated, and which have not been explored by the state-of-the-art solutions: * openness: sensors can enter or leave the system at any time without the need for any reconfiguration; * large scale: the system can be deployed in a large, urban context and ensure correct operation with a significative number of devices; * heterogeneity: the system handles different types of information without any a priori configuration. Our proposal does not require any input parameters or reconfiguration. The system can operate in open, dynamic environments such as cities, where a large number of sensing devices can appear or disappear at any time and without any prior notification. We carried out different experiments to compare the obtained results to various standard techniques to assess the validity of our proposal. We also developed a pipeline of standard techniques to produce baseline results that will be compared to those obtained by our multi-agent proposal