Revealing Robust Oil and Gas Company Macro-Strategies using Deep Multi-Agent Reinforcement Learning

The energy transition potentially poses an existential risk for major international oil companies (IOCs) if they fail to adapt to low-carbon business models. Projections of energy futures, however, are met with diverging assumptions on its scale and pace, causing disagreement among IOC decision-makers and their stakeholders over what the business model of an incumbent fossil fuel company should be. In this work, we used deep multi-agent reinforcement learning to solve an energy systems wargame wherein players simulate IOC decision-making, including hydrocarbon and low-carbon investments decisions, dividend policies, and capital structure measures, through an uncertain energy transition to explore critical and non-linear governance questions, from leveraged transitions to reserve replacements. Adversarial play facilitated by state-of-the-art algorithms revealed decision-making strategies robust to energy transition uncertainty and against multiple IOCs. In all games, robust strategies emerged in the form of low-carbon business models as a result of early transition-oriented movement. IOCs adopting such strategies outperformed business-as-usual and delayed transition strategies regardless of hydrocarbon demand projections. In addition to maximizing value, these strategies benefit greater society by contributing substantial amounts of capital necessary to accelerate the global low-carbon energy transition. Our findings point towards the need for lenders and investors to effectively mobilize transition-oriented finance and engage with IOCs to ensure responsible reallocation of capital towards low-carbon business models that would enable the emergence of fossil fuel incumbents as future low-carbon leaders

Agent-Based Computational Economics

Agent-based computational economics (ACE) is the computational study of economies modeled as evolving systems of autonomous interacting agents. Starting from initial conditions, specified by the modeler, the computational economy evolves over time as its constituent agents repeatedly interact with each other and learn from these interactions. ACE is therefore a bottom-up culture-dish approach to the study of economic systems. This study discusses the key characteristics and goals of the ACE methodology. Eight currently active research areas are highlighted for concrete illustration. Potential advantages and disadvantages of the ACE methodology are considered, along with open questions and possible directions for future research.Agent-based computational economics; Autonomous agents; Interaction networks; Learning; Evolution; Mechanism design; Computational economics; Object-oriented programming.

Power-trading in wireless communications: a cooperative networking business model

Managing the power resource in battery operated wireless devices is very crucial for extending the lifetime, here we propose the concept of power trading in wireless communications. We present a business model using sealed bid procurement auction based game theory for power-trading in cooperative wireless communication with quality of service (QoS) constraints. We formulate the problem as an auction in a buyer's market sequentially/repeatedly played with a single source and a multiple relay network. The source, in-need of cooperation of a relay due to lack of battery power to communicate with the destination, broadcasts a cooperation-request specifying its QoS requirements. The QoS that we consider here are the bit error rate and the total delay associated with relaying the source data. The relays respond with their bids in terms of Euros/bit, and the source selects the best relay based on the bids. The relays compete with each other to win the game and profit from power trading. Each relay updates its pricing index via reinforcement learning to win the game during successive bidding intervals of the repeated game. Based on this model our results show that the relay node with the best features such as a better wireless channel and a better geographical position with respect to the source and destination nodes has a better chance of winning the game, and hence giving rise to a dominant strategy. More importantly, we show that the gains from the wireless channels can be converted into economic profits which is an attractive feature of the proposed business model for power trading

Robust Learning Enabled Intelligence for the Internet-of-Things: A Survey From the Perspectives of Noisy Data and Adversarial Examples

This is the author accepted manuscript. The final version is available from IEEE via the DOI in this recordThe Internet-of-Things (IoT) has been widely adopted in a range of verticals, e.g., automation, health, energy and manufacturing. Many of the applications in these sectors, such as self-driving cars and remote surgery, are critical and high stakes applications, calling for advanced machine learning (ML) models for data analytics. Essentially, the training and testing data that are collected by massive IoT devices may contain noise (e.g., abnormal data, incorrect labels and incomplete information) and adversarial examples. This requires high robustness of ML models to make reliable decisions for IoT applications. The research of robust ML has received tremendous attentions from both academia and industry in recent years. This paper will investigate the state-of-the-art and representative works of robust ML models that can enable high resilience and reliability of IoT intelligence. Two aspects of robustness will be focused on, i.e., when the training data of ML models contains noises and adversarial examples, which may typically happen in many real-world IoT scenarios. In addition, the reliability of both neural networks and reinforcement learning framework will be investigated. Both of these two machine learning paradigms have been widely used in handling data in IoT scenarios. The potential research challenges and open issues will be discussed to provide future research directions.Engineering and Physical Sciences Research Council (EPSRC