A Provably Efficient Option-Based Algorithm for both High-Level and Low-Level Learning

Hierarchical Reinforcement Learning (HRL) approaches have shown successful results in solving a large variety of complex, structured, long-horizon problems. Nevertheless, a full theoretical understanding of this empirical evidence is currently missing. In the context of the option framework, prior research has devised efficient algorithms for scenarios where options are fixed, and the high-level policy selecting among options only has to be learned. However, the fully realistic scenario in which both the high-level and the low-level policies are learned is surprisingly disregarded from a theoretical perspective. This work makes a step towards the understanding of this latter scenario. Focusing on the finite-horizon problem, we present a meta-algorithm alternating between regret minimization algorithms instanced at different (high and low) temporal abstractions. At the higher level, we treat the problem as a Semi-Markov Decision Process (SMDP), with fixed low-level policies, while at a lower level, inner option policies are learned with a fixed high-level policy. The bounds derived are compared with the lower bound for non-hierarchical finitehorizon problems, allowing to characterize when a hierarchical approach is provably preferable, even without pre-trained options

Dynamic models for Large Eddy Simulation of compressible flows with a high order DG method

The impact of dynamic models for applications to LES of compressible flows is assessed in the framework of a numerical model based on high order discontinuous finite elements. The projections onto lower dimensional subspaces associated with lower degree basis functions are used as LES filter, along the lines proposed in Variational Multiscale templates. Comparisons with DNS results available in the literature for plane and constricted channel flows at Mach numbers 0.2, 0.7 and 1.5 show clearly that the dynamic models are able to improve the prediction of most key features of the flow with respect to the Smagorinsky models employed so far in a VMS-DG context

Green vs fossil-based energy vectors: A comparative techno-economic analysis of green ammonia and LNG value chains

This study conducts a comparative techno-economic assessment on the value chains of ammonia, as a green energy vector, and Liquefied Natural Gas (LNG), representing the benchmark energy vector, for long-distance energy transportation from Middle East to Europe. The value chain involves production from resources, conversion to an energy vector, storage and transport and reconversion of the energy vector to a suitable fuel. For comparison purposes, an electric power output of 400 MW is assumed to be produced by a power plant that utilizes either green or fossil fuels delivered to it. The adopted parameter for this comparison is the Levelized Cost of Energy (LCoE). Greenhouse gas emissions are economically penalized through the Social Cost of Carbon (SCC). Considering a SCC of 0.100 euro/kg, the LCoE of the LNG value chain is 59.19 euro/MWh, while that of ammonia is 231.71 euro/MWh. Since the cost of producing green hydrogen and purified natural gas strongly affects the results, a sensitivity analysis is performed to assess the impact of the assumed values. The SCC required to break even the LCoE of the two value chains is: 0.183 euro/MWh when considering the most favorable scenario for the green energy vector (low green hydrogen and high purified natural gas production costs) and 1.731 euro/kg when considering the most unfavorable one. This study highlights the cost-effectiveness of LNG in the current economic and regulatory landscape. However, the break-even range for the SCC indicates the potential for green ammonia to gain economic viability under higher carbon pricing scenarios

Conservative Space and Time Regularizations for the ICON Model

In this article, we consider two modified (regularized) versions of the shallow water equations which are of potential interest for the construction of global oceanic and atmospheric models. The first modified system is the Lagrangian averaged shallow water system, which involves the use of a regularized advection velocity and which has been recently proposed as a turbulence parametrization for ocean models in order to avoid an excessive damping of the computed solution. The second modified system is the pressure regularized shallow water system, which provides an alternative to traditional semi-implicit time integration schemes and which results in larger freedom in the design of the time integrator and in a better treatment of nearly geostrophic flows. The two modified systems are both nondissipative, in that they do not result in an increase of the overall dissipation of the flow. We first show how the numerical discretization of the two regularized equation sets can be constructed in a natural way within the finite difference formulation adopted for the ICON general circulation model currently under developed at the Max Planck Institute for Meteorology and at the German Weather Service. The resulting scheme is then validated on a set of idealized tests in both planar and spherical geometry, and the effects of the considered regularizations on the computed solution are analyzed concerning: stability properties and maximum allowable time steps, similarities and differences in the behavior of the solutions, discrete conservation of flow invariants such as total energy and enstrophy. Our analysis should be considered as a first step toward the use of the regularization ideas in the simulation of more complex and more realistic flows