Transport, health and climate change: Deciding on the optimal policy

Transport generates many externalities, some related to atmospheric pollution. In this paper, we focus on two: greenhouse gases, and local pollution. In the search for optimal transport policies, these two externalities have usually been analysed separately. Here, we study them jointly, in a sequential decision-making model. Our model allows for the irreversibility of the policies undertaken, as well as the possibility of a progressive reduction of uncertainties with the arrival of information. We find that when both sources of externalities are analysed jointly, structural measures enabling private transport requirements to be reduced are identified as being more advantageous economically than technological measures to reduce emissions of pollutants. We illustrate the usefulness of a joint analysis of externalities with two examples: tax measures on cars and housing policy.climate change; model of decision-making under uncertainty; irreversibilities; transport policy

Stochastic Optimization of Bioreactor Control Policies Using a Markov Decision Process Model

Biopharmaceuticals are the fastest-growing segment of the pharmaceutical industry. Their manufacture is complicated by the uncertainty exhibited therein. Scholars have studied the planning and operation of such production systems under some uncertainties, but the simultaneous consideration of fermentation and resin yield uncertainty is lacking so-far. To study the optimal operation of biopharmaceutical production and purification systems under these uncertainties, a stochastic, dynamic approach is necessary. This thesis provides such a model by extending an existing discrete state-space, infinite horizon Markov decision process model of upstream fermentation. Tissue Plasminogen Activator fermentation and chromatography was implemented. This example was used to discuss the optimal policy for operating different fermentation setups. The average per-cycle operating profit of a serial setup was 1,272 $; the parallel setup produced negative average rewards. Managerial insights were derived from a comparison to a basic, titer maximizing policy and process sensitivities. In conclusion, the integrated stochastic optimization of biopharma production and purification control aids decision making. However, the model assumptions pose room for further studies. Keywords: Markov decision process; biopharmaceuticals production; fermentation uncertainty; chromatography resin; stochastic performance decay

Unexpected Management Choices When Accounting for Uncertainty in Ecosystem Service Tradeoff Analyses

Resource management and conservation increasingly focus on ecosystem service provisioning and potential tradeoffs among services under different management actions. Application of bioeconomic approaches to tradeoffs assessment is touted as a way to find win-win outcomes or avoid unnecessary stakeholder conflict. Yet, nearly all assessments to date have ignored inherent uncertainties in the provision and valuation of services. We incorporate uncertainty into the ecosystem services analytical framework and show how such inclusion improves optimal decision making. In particular, we show: (1) “suboptimal” solutions can become optimal when uncertainties are accounted for; (2) uncertainty paradoxically makes stakeholders value conservation despite their lack of preference for it; and (3) substantial losses or missed gains in ecosystem service provisioning can be incurred when uncertainty is ignored. Our results highlight the urgency of accounting for uncertainties in ecosystem services in tradeoff assessments given the widespread use of this approach by government agencies and conservation organizations

Combined sourcing and inventory management using capacity reservation and spot market

Leading companies in several industries purchase materials with the combined use of capacity reservation contracts and spot market. We analyse the optimal and a simplified policy for making long-term capacity reservation and periodic ordering/inventory decisions using the above two sources under stochastic demand and random spot market price fluctuations. In a numerical study we assess the effects of demand and spot market price uncertainties and of other parameters on both the optimal and simplified policy. We provide insights into the interaction of capacity reservation decision, demand uncertainty induced safety inventory, and inventory resulting from forward buying on the spot marke

Optimal integrated sizing and operation of a CHP system with Monte Carlo risk analysis for long-term uncertainty in energy demands

In this study a probabilistic approach for optimal sizing of cogeneration systems under long-term uncertainty in energy demand is proposed. A dynamic simulation framework for detailed modeling of the energy system is defined, consisting in both traditional and optimal operational strategies evaluation. A two-stage stochastic optimization algorithm is developed, adopting Monte Carlo method for the definition of a multi-objective optimization problem. An Italian hospital facility has been used as a case study and a gas internal combustion engine is considered for the cogeneration unit. The results reveal that the influence of uncertainties on both optimal size and annual total cost is significant. Optimal size obtained with the traditional deterministic approach are found to be sub-optimal (up to 30% larger) and the predicted annual cost saving is always lower when accounting for uncertainties. Pareto frontiers of different CHP configurations are presented and show the effectiveness of the proposed method as a useful tool for risk management and focused decision-making, as tradeoffs between system efficiency and system robustness

Optimal Decision-Making under Uncertainty - Application to Power Transmission Investments

Economists define investment as the act of incurring immediate costs with the expectation of future returns. An investment project, as every asset has a value. For successfully investing in and managing these assets is crucial not only recognizing what the value is but also the sources of this value. Most investment decisions share three characteristics in different degrees. First, investments are partially or totally irreversible. Roughly speaking, the initial investment cost is at least partially sunk; i.e. it is impossible to recover all the expenditures if the decision-maker changes his mind. Second, there is uncertainty in the revenues from the investment, and therefore, risk associated with this. Third, all decision-making has some leeway about the timing of the investment. It is possible to defer the decision making to get more information about the future. These three features interact to determine the optimal decisions of investors on a given investment project. Transmission utilities are faced with investment projects, which hold these three characteristics: irreversibility, uncertainty and the choice of timing. In this context, an efficient decision making process is, therefore, based on managing the uncertainties and understanding the relationships between risks and opportunities in order to achieve a well-timed investment execution. Therefore, strategic flexibility for seizing opportunities and cutting losses contingent upon the market evolution is of huge value. Strategic flexibility is a risk management method that is gaining ongoing research attention as it enables properly managing major uncertainties, which are unsolved at the time of making decisions. Hence, valuing added flexibility in transmission investment portfolios, for instance, by investing in power electronic-based controller meanwhile transmission line projects are deferred, is necessary to make optimal network upgrading. Nevertheless, expressing the value of flexibility in economic terms is not a trivial task and requires new, sophisticated valuing tools, since the traditional investment theory has not recognized the important implications of the interaction between the three aforementioned investment features. Any attempt to quantify investment flexibility almost naturally leads to the concept of Real Options (RO). The RO technique provides a well-founded framework –based on the theory of financial options, and consequently, stochastic dynamic programming- to assess strategic investments under uncertainty. In the first RO applications, valuation was normally confined to the investment options that can be easily assimilated to financial options, for which solutions are well-known and readily available. Nevertheless, an investor confront with a diverse set of opportunities. From this point of view, investment projects can be seen as a portfolio of options, where its value is driven by several stochastic variables. The introduction of multiple interacting options into real options models highly increases the problem complexity, making traditional numerical approaches impracticable. However in the recent years, simulation procedures for solving multiple American options have been successfully proposed. One of the most promising approaches is the Least Square Monte Carlo (LSM) method proposed by Longstaff and Schwartz in 2001. LSM method is based on stochastic chronological simulation and uses least squares linear regression to determine the optimal stopping time (optimal path) in the decision making process. This chapter lays out a general background about key concepts -uncertainty and risk- and the most usual risk management techniques in transmission investment are provided. Then, the concept of strategic flexibility is introduced in order to set its ability for dealing with the uncertainties involved in the investment problem. In addition, new criteria and advantages of ROV approach compared with classical probabilistic choice are presented, by exposing a LSM-based method for decomposing and evaluating the complex real option problem involved in flexible transmission investments under uncertainties. The proposed methodology is applied in a study case which evaluates an interconnection reinforcement on the European interconnected power system, by showing how the valuation of flexibility is a key task for making efficient and well-timed investments in the transmission network. The impact of two network upgrades on the system-wide welfare is analyzed. These upgrades are the development of a new interconnected line and the installation of a power electronic-based controller. Both upgrades represent measures to strengthen the German-Dutch interconnections due to the fact that these are among the most important corridors within the Central Western European (CWE) region. Hence, an interconnection project, which is currently under study, is compared to flexible investment in order to shed some light on the influence of the strategic flexibility on the optimal decision-making process. The research is focused on assessing the impact of different wind power in-feed scenarios in detail as well as the uncertainty of the demand growth, generation cost evolution and the installed wind capacity on the decision-making process. The presented approach might serve as a basis for a decision-making tool for regulatory agencies in order to quantify the necessity for network upgrades.Fil: Blanco, Gerardo. Universidad Nacional de Asunción; ParaguayFil: Olsina, Fernando Gabriel. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - San Juan; Argentina. Universidad Nacional de San Juan. Facultad de Ingeniería. Instituto de Energía Eléctrica; Argentin

Augmented Human Machine Intelligence for Distributed Inference

With the advent of the internet of things (IoT) era and the extensive deployment of smart devices and wireless sensor networks (WSNs), interactions of humans and machine data are everywhere. In numerous applications, humans are essential parts in the decision making process, where they may either serve as information sources or act as the final decision makers. For various tasks including detection and classification of targets, detection of outliers, generation of surveillance patterns and interactions between entities, seamless integration of the human and the machine expertise is required where they simultaneously work within the same modeling environment to understand and solve problems. Efficient fusion of information from both human and sensor sources is expected to improve system performance and enhance situational awareness. Such human-machine inference networks seek to build an interactive human-machine symbiosis by merging the best of the human with the best of the machine and to achieve higher performance than either humans or machines by themselves. In this dissertation, we consider that people often have a number of biases and rely on heuristics when exposed to different kinds of uncertainties, e.g., limited information versus unreliable information. We develop novel theoretical frameworks for collaborative decision making in complex environments when the observers may include both humans and physics-based sensors. We address fundamental concerns such as uncertainties, cognitive biases in human decision making and derive human decision rules in binary decision making. We model the decision-making by generic humans working in complex networked environments that feature uncertainties, and develop new approaches and frameworks facilitating collaborative human decision making and cognitive multi-modal fusion. The first part of this dissertation exploits the behavioral economics concept Prospect Theory to study the behavior of human binary decision making under cognitive biases. Several decision making systems involving humans\u27 participation are discussed, and we show the impact of human cognitive biases on the decision making performance. We analyze how heterogeneity could affect the performance of collaborative human decision making in the presence of complex correlation relationships among the behavior of humans and design the human selection strategy at the population level. Next, we employ Prospect Theory to model the rationality of humans and accurately characterize their behaviors in answering binary questions. We design a weighted majority voting rule to solve classification problems via crowdsourcing while considering that the crowd may include some spammers. We also propose a novel sequential task ordering algorithm to improve system performance for classification in crowdsourcing composed of unreliable human workers. In the second part of the dissertation, we study the behavior of cognitive memory limited humans in binary decision making and develop efficient approaches to help memory constrained humans make better decisions. We show that the order in which information is presented to the humans impacts their decision making performance. Next, we consider the selfish behavior of humans and construct a unified incentive mechanism for IoT based inference systems while addressing the selfish concerns of the participants. We derive the optimal amount of energy that a selfish sensor involved in the signal detection task must spend in order to maximize a certain utility function, in the presence of buyers who value the result of signal detection carried out by the sensor. Finally, we design a human-machine collaboration framework that blends both machine observations and human expertise to solve binary hypothesis testing problems semi-autonomously. In networks featuring human-machine teaming/collaboration, it is critical to coordinate and synthesize the operations of the humans and machines (e.g., robots and physical sensors). Machine measurements affect human behaviors, actions, and decisions. Human behavior defines the optimal decision-making algorithm for human-machine networks. In today\u27s era of artificial intelligence, we not only aim to exploit augmented human-machine intelligence to ensure accurate decision making; but also expand intelligent systems so as to assist and improve such intelligence

Deep VULMAN: A Deep Reinforcement Learning-Enabled Cyber Vulnerability Management Framework

Cyber vulnerability management is a critical function of a cybersecurity operations center (CSOC) that helps protect organizations against cyber-attacks on their computer and network systems. Adversaries hold an asymmetric advantage over the CSOC, as the number of deficiencies in these systems is increasing at a significantly higher rate compared to the expansion rate of the security teams to mitigate them in a resource-constrained environment. The current approaches are deterministic and one-time decision-making methods, which do not consider future uncertainties when prioritizing and selecting vulnerabilities for mitigation. These approaches are also constrained by the sub-optimal distribution of resources, providing no flexibility to adjust their response to fluctuations in vulnerability arrivals. We propose a novel framework, Deep VULMAN, consisting of a deep reinforcement learning agent and an integer programming method to fill this gap in the cyber vulnerability management process. Our sequential decision-making framework, first, determines the near-optimal amount of resources to be allocated for mitigation under uncertainty for a given system state and then determines the optimal set of prioritized vulnerability instances for mitigation. Our proposed framework outperforms the current methods in prioritizing the selection of important organization-specific vulnerabilities, on both simulated and real-world vulnerability data, observed over a one-year period.Comment: 12 pages, 3 figure