Reinforcement Learning, Intelligent Control and their Applications in Connected and Autonomous Vehicles

Reinforcement learning (RL) has attracted large attention over the past few years. Recently, we developed a data-driven algorithm to solve predictive cruise control (PCC) and games output regulation problems. This work integrates our recent contributions to the application of RL in game theory, output regulation problems, robust control, small-gain theory and PCC. The algorithm was developed for $H_\infty$ adaptive optimal output regulation of uncertain linear systems, and uncertain partially linear systems to reject disturbance and also force the output of the systems to asymptotically track a reference. In the PCC problem, we determined the reference velocity for each autonomous vehicle in the platoon using the traffic information broadcasted from the lights to reduce the vehicles\u27 trip time. Then we employed the algorithm to design an approximate optimal controller for the vehicles. This controller is able to regulate the headway, velocity and acceleration of each vehicle to the desired values. Simulation results validate the effectiveness of the algorithms

Active sensor fault tolerant output feedback tracking control for wind turbine systems via T-S model

This paper presents a new approach to active sensor fault tolerant tracking control (FTTC) for offshore wind turbine (OWT) described via Takagi–Sugeno (T–S) multiple models. The FTTC strategy is designed in such way that aims to maintain nominal wind turbine controller without any change in both fault and fault-free cases. This is achieved by inserting T–S proportional state estimators augmented with proportional and integral feedback (PPI) fault estimators to be capable to estimate different generators and rotor speed sensors fault for compensation purposes. Due to the dependency of the FTTC strategy on the fault estimation the designed observer has the capability to estimate a wide range of time varying fault signals. Moreover, the robustness of the observer against the difference between the anemometer wind speed measurement and the immeasurable effective wind speed signal has been taken into account. The corrected measurements fed to a T–S fuzzy dynamic output feedback controller (TSDOFC) designed to track the desired trajectory. The stability proof with H∞ performance and D-stability constraints is formulated as a Linear Matrix Inequality (LMI) problem. The strategy is illustrated using a non-linear benchmark system model of a wind turbine offered within a competition led by the companies Mathworks and KK-Electronic

Valuing adaptation under rapid change

AbstractThe methods used to plan adaptation to climate change have been heavily influenced by scientific narratives of gradual change and economic narratives of marginal adjustments to that change. An investigation of the theoretical aspects of how the climate changes suggests that scientific narratives of climate change are socially constructed, biasing scientific narratives to descriptions of gradual as opposed rapid, non-linear change. Evidence of widespread step changes in recent climate records and in model projections of future climate is being overlooked because of this. Step-wise climate change has the potential to produce rapid increases in extreme events that can cross institutional, geographical and sectoral domains.Likewise, orthodox economics is not well suited to the deep uncertainty faced under climate change, requiring a multi-faceted approach to adaptation. The presence of tangible and intangible values range across five adaptation clusters: goods; services; capital assets and infrastructure; social assets and infrastructure; and natural assets and infrastructure. Standard economic methods have difficulty in giving adequate weight to the different types of values across these clusters. They also do not account well for the inter-connectedness of impacts and subsequent responses between agents in the economy. As a result, many highly-valued aspects of human and environmental capital are being overlooked.Recent extreme events are already pressuring areas of public policy, and national strategies for emergency response and disaster risk reduction are being developed as a consequence. However, the potential for an escalation of total damage costs due to rapid change requires a coordinated approach at the institutional level, involving all levels of government, the private sector and civil society.One of the largest risks of maladaptation is the potential for un-owned risks, as risks propagate across domains and responsibility for their management is poorly allocated between public and private interests, and between the roles of the individual and civil society. Economic strategies developed by the disaster community for disaster response and risk reduction provide a base to work from, but many gaps remain.We have developed a framework for valuing adaptation that has the following aspects: the valuation of impacts thus estimating values at risk, the evaluation of different adaptation options and strategies based on cost, and the valuation of benefits expressed as a combination of the benefits of avoided damages and a range of institutional values such as equity, justice, sustainability and profit.The choice of economic methods and tools used to assess adaptation depends largely on the ability to constrain uncertainty around problems (predictive uncertainty) and solutions (outcome uncertainty). Orthodox methods can be used where both are constrained, portfolio methodologies where problems are constrained and robust methodologies where solutions are constrained. Where both are unconstrained, process-based methods utilising innovation methods and adaptive management are most suitable. All methods should involve stakeholders where possible.Innovative processes methods that enable transformation will be required in some circumstances, to allow institutions, sectors and communities to prepare for anticipated major change.Please cite this report as: Jones, RN, Young, CK, Handmer, J, Keating, A, Mekala, GD, Sheehan, P 2013 Valuing adaptation under rapid change, National Climate Change Adaptation Research Facility, Gold Coast, pp. 192.The methods used to plan adaptation to climate change have been heavily influenced by scientific narratives of gradual change and economic narratives of marginal adjustments to that change. An investigation of the theoretical aspects of how the climate changes suggests that scientific narratives of climate change are socially constructed, biasing scientific narratives to descriptions of gradual as opposed rapid, non-linear change. Evidence of widespread step changes in recent climate records and in model projections of future climate is being overlooked because of this. Step-wise climate change has the potential to produce rapid increases in extreme events that can cross institutional, geographical and sectoral domains.Likewise, orthodox economics is not well suited to the deep uncertainty faced under climate change, requiring a multi-faceted approach to adaptation. The presence of tangible and intangible values range across five adaptation clusters: goods; services; capital assets and infrastructure; social assets and infrastructure; and natural assets and infrastructure. Standard economic methods have difficulty in giving adequate weight to the different types of values across these clusters. They also do not account well for the inter-connectedness of impacts and subsequent responses between agents in the economy. As a result, many highly-valued aspects of human and environmental capital are being overlooked.Recent extreme events are already pressuring areas of public policy, and national strategies for emergency response and disaster risk reduction are being developed as a consequence. However, the potential for an escalation of total damage costs due to rapid change requires a coordinated approach at the institutional level, involving all levels of government, the private sector and civil society.One of the largest risks of maladaptation is the potential for un-owned risks, as risks propagate across domains and responsibility for their management is poorly allocated between public and private interests, and between the roles of the individual and civil society. Economic strategies developed by the disaster community for disaster response and risk reduction provide a base to work from, but many gaps remain.We have developed a framework for valuing adaptation that has the following aspects: the valuation of impacts thus estimating values at risk, the evaluation of different adaptation options and strategies based on cost, and the valuation of benefits expressed as a combination of the benefits of avoided damages and a range of institutional values such as equity, justice, sustainability and profit.The choice of economic methods and tools used to assess adaptation depends largely on the ability to constrain uncertainty around problems (predictive uncertainty) and solutions (outcome uncertainty). Orthodox methods can be used where both are constrained, portfolio methodologies where problems are constrained and robust methodologies where solutions are constrained. Where both are unconstrained, process-based methods utilising innovation methods and adaptive management are most suitable. All methods should involve stakeholders where possible.Innovative processes methods that enable transformation will be required in some circumstances, to allow institutions, sectors and communities to prepare for anticipated major change

Optimal adaptive control of time-delay dynamical systems with known and uncertain dynamics

Delays are found in many industrial pneumatic and hydraulic systems, and as a result, the performance of the overall closed-loop system deteriorates unless they are explicitly accounted. It is also possible that the dynamics of such systems are uncertain. On the other hand, optimal control of time-delay systems in the presence of known and uncertain dynamics by using state and output feedback is of paramount importance. Therefore, in this research, a suite of novel optimal adaptive control (OAC) techniques are undertaken for linear and nonlinear continuous time-delay systems in the presence of uncertain system dynamics using state and/or output feedback. First, the optimal regulation of linear continuous-time systems with state and input delays by utilizing a quadratic cost function over infinite horizon is addressed using state and output feedback. Next, the optimal adaptive regulation is extended to uncertain linear continuous-time systems under a mild assumption that the bounds on system matrices are known. Subsequently, the event-triggered optimal adaptive regulation of partially unknown linear continuous time systems with state-delay is addressed by using integral reinforcement learning (IRL). It is demonstrated that the optimal control policy renders asymptotic stability of the closed-loop system provided the linear time-delayed system is controllable and observable. The proposed event-triggered approach relaxed the need for continuous availability of state vector and proven to be zeno-free. Finally, the OAC using IRL neural network based control of uncertain nonlinear time-delay systems with input and state delays is investigated. An identifier is proposed for nonlinear time-delay systems to approximate the system dynamics and relax the need for the control coefficient matrix in generating the control policy. Lyapunov analysis is utilized to design the optimal adaptive controller, derive parameter/weight tuning law and verify stability of the closed-loop system”--Abstract, page iv

Output-feedback online optimal control for a class of nonlinear systems

In this paper an output-feedback model-based reinforcement learning (MBRL) method for a class of second-order nonlinear systems is developed. The control technique uses exact model knowledge and integrates a dynamic state estimator within the model-based reinforcement learning framework to achieve output-feedback MBRL. Simulation results demonstrate the efficacy of the developed method

Environmental Efficiency, Emission Trends and Labour Productivity: Trade-Off or Joint Dynamics? Empirical Evidence Using NAMEA Panel Data

The paper provides new empirical evidence on the relationship between environmental efficiency and labour productivity using industry level data. We first provide a critical and extensive discussion around the interconnected issues of environmental efficiency and performance, firm performances and labour productivity, and environmental and non-environmental innovation dynamics. The most recent literature dealing with environmental innovation, environmental regulations and economic performances is taken as reference. We then test a newly adapted EKC hypothesis, by verifying the correlation between the two trends of environmental efficiency (productivity, namely sector emission on added value) and labour productivity (added value on employees) over a dynamic path. We exploit official NAMEA data sources for Italy over 1990-2002 for 29 sectoral branches. The period is crucial since environmental issues and then environmental policies came into the arena, and a restructuring of the economy occurred. It is thus interesting to assess the extent to which capital investments for the economy as a whole are associated with a positive or negative correlation between environmental efficiency of productive branches and labour productivity, often claimed by mainstream theory dealing with innovation in environmental economics. We believe that on the basis of the theoretical and empirical analyses focusing on innovation paths, firm performances and environmental externalities, there are good reasons to expect a positive correlation between environmental and labour productivities, or in alternative terms a negative correlation between mission intensity of production and labour productivity. The tested hypothesis is crucial within the long standing discussion over the potential trade-off or complementarity between environmental and labour productivity, strictly associated with sectoral and national technological innovation paths. The main added value of the paper is the analysis of the aforementioned hypothesis by exploiting a panel data set based on official NAMEA sectoral disaggregated accounting data, providing both cross section heterogeneity and a sufficient time span. We find that for most emissions, if not all, a negative correlation emerges between labour productivity and environmental productivity. Though this trend appears driven by the macro sectors services, manufacturing and industry, this evidence is not homogenous across emissions. In some cases U-shapes arise, mainly for services, and the assessment of Turning Points is crucial. Manufacturing and industry, all in all, seem to have a stronger weight. Overall, then, labour productivity dynamics seem to be complementary to a decreasing emission intensity of productive processes. The extent to which this evidence derives from endogenous market forces, industrial restructuring and/or from policy effects is scope for further research. The relative role of manufacturing and services in explaining this pattern is also to be analysed in future empirical analyses. In addition, the role of capital stocks and trade openness are extensions which may add value to future analyses carried out on the same NAMEA dataset.Decoupling, NAMEA Emissions, Labour Productivity, Sectoral Added Value, Kuznets Curves, Environmental Efficiency