On Uncertainty in Deep State Space Models for Model-Based Reinforcement Learning

Improved state space models, such as Recurrent State Space Models (RSSMs), are a key factor behind recent advances in model-based reinforcement learning (RL). Yet, despite their empirical success, many of the underlying design choices are not well understood. We show that RSSMs use a suboptimal inference scheme and that models trained using this inference overestimate the aleatoric uncertainty of the ground truth system. We find this overestimation implicitly regularizes RSSMs and allows them to succeed in model-based RL. We postulate that this implicit regularization fulfills the same functionality as explicitly modeling epistemic uncertainty, which is crucial for many other model-based RL approaches. Yet, overestimating aleatoric uncertainty can also impair performance in cases where accurately estimating it matters, e.g., when we have to deal with occlusions, missing observations, or fusing sensor modalities at different frequencies. Moreover, the implicit regularization is a side-effect of the inference scheme and not the result of a rigorous, principled formulation, which renders analyzing or improving RSSMs difficult. Thus, we propose an alternative approach building on well-understood components for modeling aleatoric and epistemic uncertainty, dubbed Variational Recurrent Kalman Network (VRKN). This approach uses Kalman updates for exact smoothing inference in a latent space and Monte Carlo Dropout to model epistemic uncertainty. Due to the Kalman updates, the VRKN can naturally handle missing observations or sensor fusion problems with varying numbers of observations per time step. Our experiments show that using the VRKN instead of the RSSM improves performance in tasks where appropriately capturing aleatoric uncertainty is crucial while matching it in the deterministic standard benchmarks

Towards A Double-Edged Sword: Modelling the Impact in Agile Software Development

Agile methods are state of the art in software development. Companies worldwide apply agile to counter the dynamics of the markets. We know, that various factors like culture influence the successfully application of agile methods in practice and the sucess is differing from company to company. To counter these problems, we combine two causal models presented in literature: The Agile Practices Impact Model and the Model of Cultural Impact. In this paper, we want to better understand the two facets of factors in agile: Those influencing their application and those impacting the results when applying them. This papers core contribution is the Agile Influence and Imact Model, describing the factors influencing agile elements and the impact on specific characteristics in a systematic manner

Information-Theoretic Trust Regions for Stochastic Gradient-Based Optimization

Stochastic gradient-based optimization is crucial to optimize neural networks. While popular approaches heuristically adapt the step size and direction by rescaling gradients, a more principled approach to improve optimizers requires second-order information. Such methods precondition the gradient using the objective’s Hessian. Yet, computing the Hessian is usually expensive and effectively using second-order information in the stochastic gradient setting is non-trivial. We propose using Information-Theoretic Trust Region Optimization (arTuRO) for improved updates with uncertain second-order information. By modeling the network parameters as a Gaussian distribution and using a Kullback-Leibler divergence-based trust region, our approach takes bounded steps accounting for the objective’s curvature and uncertainty in the parameters. Before each update, it solves the trust region problem for an optimal step size, resulting in a more stable and faster optimization process. We approximate the diagonal elements of the Hessian from stochastic gradients using a simple recursive least squares approach, constructing a model of the expected Hessian over time using only first-order information. We show that arTuRO combines the fast convergence of adaptive moment-based optimization with the generalization capabilities of SGD

Information-Theoretic Trust Regions for Stochastic Gradient-Based Optimization

Stochastic gradient-based optimization is crucial to optimize neural networks. While popular approaches heuristically adapt the step size and direction by rescaling gradients, a more principled approach to improve optimizers requires second-order information. Such methods precondition the gradient using the objective's Hessian. Yet, computing the Hessian is usually expensive and effectively using second-order information in the stochastic gradient setting is non-trivial. We propose using Information-Theoretic Trust Region Optimization (arTuRO) for improved updates with uncertain second-order information. By modeling the network parameters as a Gaussian distribution and using a Kullback-Leibler divergence-based trust region, our approach takes bounded steps accounting for the objective's curvature and uncertainty in the parameters. Before each update, it solves the trust region problem for an optimal step size, resulting in a more stable and faster optimization process. We approximate the diagonal elements of the Hessian from stochastic gradients using a simple recursive least squares approach, constructing a model of the expected Hessian over time using only first-order information. We show that arTuRO combines the fast convergence of adaptive moment-based optimization with the generalization capabilities of SGD

Hydrogen refueling station networks for heavy-duty vehicles in future power systems

A potential solution to reduce greenhouse gas (GHG) emissions in the transport sector is to use alternatively fueled vehicles (AFV). Heavy-duty vehicles (HDV) emit a large share of GHG emissions in the transport sector and are therefore the subject of growing attention from global regulators. Fuel cell and green hydrogen technologies are a promising option to decarbonize HDVs, as their fast refueling and long vehicle ranges are consistent with current logistic operational requirements. Moreover, the application of green hydrogen in transport could enable more effective integration of renewable energies (RE) across different energy sectors. This paper explores the interplay between HDV Hydrogen Refueling Stations (HRS) that produce hydrogen locally and the power system by combining an infrastructure location planning model and an electricity system optimization model that takes grid expansion options into account. Two scenarios – one sizing refueling stations to support the power system and one sizing them independently of it – are assessed regarding their impacts on the total annual electricity system costs, regional RE integration and the levelized cost of hydrogen (LCOH). The impacts are calculated based on locational marginal pricing for 2050. Depending on the integration scenario, we find average LCOH of between 4.83 euro/kg and 5.36 euro/kg, for which nodal electricity prices are the main determining factor as well as a strong difference in LCOH between north and south Germany. Adding HDV-HRS incurs power transmission expansion as well as higher power supply costs as the total power demand increases. From a system perspective, investing in HDV-HRS in symbiosis with the power system rather than independently promises cost savings of around seven billion euros per annum. We therefore conclude that the co-optimization of multiple energy sectors is important for investment planning and has the potential to exploit synergies

Stellungsnahme zum Antrag "Wirtschaftspolitische Kehrtwende endlich einleiten"

Stellungnahme zur Anhörung des Ausschusses für Wirtschaft, Energie, Industrie, Mittelstand und Handwerk am 7. September 2016 zum Antrag der Fraktion der CDU und der Fraktion der FDP. 31. August 201

A Pay-as-Bid Mechanism for Pricing Utility Computing

Encountering the increasing demand for high-performance computational resources in academic as well as commercial organisations, utility computing offers a solution by providing users with on-demand availability of requested computing services. Approaches to the fundamental issue of resource allocation include the use of technical scheduling mechanisms as well as introducing economic ideas into the allocation schemes. Technical scheduling mechanisms are often very simple (such as first-in-first-out) but suffer under the shortcoming to adequately prioritize jobs in times when demand exceeds supply. As empirical studies show, Grids (such as PlanetLab) are frequently characterized by huge excess demand for resources. This is where economic models such as markets come into play. Hitherto, market mechanisms are either (too) simple or too complex for usage in Grids. The contribution of this paper is threefold. Firstly, a mechanism for Grids is proposed, which is still simple but geared up for use in the Grid. Secondly the mechanism is embedded in state-of-the-art Grid middleware Sun N1 Grid Engine 6. Thirdly, it is shown by means of a numerical case study that this mechanism is superior to other commonly used mechanisms