Multi-agent quality of experience control

In the framework of the Future Internet, the aim of the Quality of Experience (QoE) Control functionalities is to track the personalized desired QoE level of the applications. The paper proposes to perform such a task by dynamically selecting the most appropriate Classes of Service (among the ones supported by the network), this selection being driven by a novel heuristic Multi-Agent Reinforcement Learning (MARL) algorithm. The paper shows that such an approach offers the opportunity to cope with some practical implementation problems: in particular, it allows to face the so-called “curse of dimensionality” of MARL algorithms, thus achieving satisfactory performance results even in the presence of several hundreds of Agents

A control theoretic approach for security of cyber-physical systems

In this dissertation, several novel defense methodologies for cyber-physical systems have been proposed. First, a special type of cyber-physical system, the RFID system, is considered for which a lightweight mutual authentication and ownership management protocol is proposed in order to protect the data confidentiality and integrity. Then considering the fact that the protection of the data confidentiality and integrity is insufficient to guarantee the security in cyber-physical systems, we turn to the development of a general framework for developing security schemes for cyber-physical systems wherein the cyber system states affect the physical system and vice versa. After that, we apply this general framework by selecting the traffic flow as the cyber system state and a novel attack detection scheme that is capable of capturing the abnormality in the traffic flow in those communication links due to a class of attacks has been proposed. On the other hand, an attack detection scheme that is capable of detecting both sensor and actuator attacks is proposed for the physical system in the presence of network induced delays and packet losses. Next, an attack detection scheme is proposed when the network parameters are unknown by using an optimal Q-learning approach. Finally, this attack detection and accommodation scheme has been further extended to the case where the network is modeled as a nonlinear system with unknown system dynamics --Abstract, page iv

Physics-Informed Machine Learning for Data Anomaly Detection, Classification, Localization, and Mitigation: A Review, Challenges, and Path Forward

Advancements in digital automation for smart grids have led to the installation of measurement devices like phasor measurement units (PMUs), micro-PMUs ($\mu$-PMUs), and smart meters. However, a large amount of data collected by these devices brings several challenges as control room operators need to use this data with models to make confident decisions for reliable and resilient operation of the cyber-power systems. Machine-learning (ML) based tools can provide a reliable interpretation of the deluge of data obtained from the field. For the decision-makers to ensure reliable network operation under all operating conditions, these tools need to identify solutions that are feasible and satisfy the system constraints, while being efficient, trustworthy, and interpretable. This resulted in the increasing popularity of physics-informed machine learning (PIML) approaches, as these methods overcome challenges that model-based or data-driven ML methods face in silos. This work aims at the following: a) review existing strategies and techniques for incorporating underlying physical principles of the power grid into different types of ML approaches (supervised/semi-supervised learning, unsupervised learning, and reinforcement learning (RL)); b) explore the existing works on PIML methods for anomaly detection, classification, localization, and mitigation in power transmission and distribution systems, c) discuss improvements in existing methods through consideration of potential challenges while also addressing the limitations to make them suitable for real-world applications

Critical Market Crashes

This review is a partial synthesis of the book ``Why stock market crash'' (Princeton University Press, January 2003), which presents a general theory of financial crashes and of stock market instabilities that his co-workers and the author have developed over the past seven years. The study of the frequency distribution of drawdowns, or runs of successive losses shows that large financial crashes are ``outliers'': they form a class of their own as can be seen from their statistical signatures. If large financial crashes are ``outliers'', they are special and thus require a special explanation, a specific model, a theory of their own. In addition, their special properties may perhaps be used for their prediction. The main mechanisms leading to positive feedbacks, i.e., self-reinforcement, such as imitative behavior and herding between investors are reviewed with many references provided to the relevant literature outside the confine of Physics. Positive feedbacks provide the fuel for the development of speculative bubbles, preparing the instability for a major crash. We demonstrate several detailed mathematical models of speculative bubbles and crashes. The most important message is the discovery of robust and universal signatures of the approach to crashes. These precursory patterns have been documented for essentially all crashes on developed as well as emergent stock markets, on currency markets, on company stocks, and so on. The concept of an ``anti-bubble'' is also summarized, with two forward predictions on the Japanese stock market starting in 1999 and on the USA stock market still running. We conclude by presenting our view of the organization of financial markets.Comment: Latex 89 pages and 38 figures, in press in Physics Report

Coherent Price Systems and Uncertainty-Neutral Valuation

We consider fundamental questions of arbitrage pricing arising when the uncertainty model is given by a set of possible mutually singular probability measures. With a single probability model, essential equivalence between the absence of arbitrage and the existence of an equivalent martingale measure is a folk theorem, see Harrison and Kreps (1979). We establish a microeconomic foundation of sublinear price systems and present an extension result. In this context we introduce a prior dependent notion of marketed spaces and viable price systems. We associate this extension with a canonically altered concept of equivalent symmetric martingale measure sets, in a dynamic trading framework under absence of prior depending arbitrage. We prove the existence of such sets when volatility uncertainty is modeled by a stochastic differential equation, driven by Peng's G-Brownian motions

Measuring time preferences

We review research that measures time preferences—i.e., preferences over intertemporal tradeoffs. We distinguish between studies using financial flows, which we call “money earlier or later” (MEL) decisions and studies that use time-dated consumption/effort. Under different structural models, we show how to translate what MEL experiments directly measure (required rates of return for financial flows) into a discount function over utils. We summarize empirical regularities found in MEL studies and the predictive power of those studies. We explain why MEL choices are driven in part by some factors that are distinct from underlying time preferences.National Institutes of Health (NIA R01AG021650 and P01AG005842) and the Pershing Square Fund for Research in the Foundations of Human Behavior