Closing the loop between neural network simulators and the OpenAI Gym

Since the enormous breakthroughs in machine learning over the last decade, functional neural network models are of growing interest for many researchers in the field of computational neuroscience. One major branch of research is concerned with biologically plausible implementations of reinforcement learning, with a variety of different models developed over the recent years. However, most studies in this area are conducted with custom simulation scripts and manually implemented tasks. This makes it hard for other researchers to reproduce and build upon previous work and nearly impossible to compare the performance of different learning architectures. In this work, we present a novel approach to solve this problem, connecting benchmark tools from the field of machine learning and state-of-the-art neural network simulators from computational neuroscience. This toolchain enables researchers in both fields to make use of well-tested high-performance simulation software supporting biologically plausible neuron, synapse and network models and allows them to evaluate and compare their approach on the basis of standardized environments of varying complexity. We demonstrate the functionality of the toolchain by implementing a neuronal actor-critic architecture for reinforcement learning in the NEST simulator and successfully training it on two different environments from the OpenAI Gym

Non-constructive interval simulation of dynamic systems

Publisher PD

PGNME: A Domain Decomposition Algorithm for Distributed Power System Dynamic Simulation on High Performance Computing Platforms

Dynamic simulation of a large-scale electric power system involves solving a large number of differential algebraic equations (DAEs) every simulation time-step. With the ever-growing size and complexity of power grid, dynamic simulation becomes more and more time-consuming and computationally difficult using conventional sequential simulation techniques. This thesis presents a fully distributed approach intended for implementation on High Performance Computer (HPC) clusters. A novel, relaxation-based domain decomposition algorithm known as Parallel-General-Norton with Multiple-port Equivalent (PGNME) is proposed as the core technique of a two-stage decomposition approach to divide the overall dynamic simulation problem into a set of sub problems that can be solved concurrently. While the convergence property has traditionally been a concern for relaxation-based decomposition, an estimation mechanism based on multiple-port network equivalent is adopted as the preconditioner to enhance the convergence of the proposed algorithm. The algorithm is presented in detail and validated both in terms of accuracy and capabilit

PGNME: A Domain Decomposition Algorithm for Distributed Power System Dynamic Simulation on High Performance Computing Platforms

Dynamic simulation of a large-scale electric power system involves solving a large number of differential algebraic equations (DAEs) every simulation time-step. With the ever-growing size and complexity of power grid, dynamic simulation becomes more and more time-consuming and computationally difficult using conventional sequential simulation techniques. This thesis presents a fully distributed approach intended for implementation on High Performance Computer (HPC) clusters. A novel, relaxation-based domain decomposition algorithm known as Parallel-General-Norton with Multiple-port Equivalent (PGNME) is proposed as the core technique of a two-stage decomposition approach to divide the overall dynamic simulation problem into a set of sub problems that can be solved concurrently. While the convergence property has traditionally been a concern for relaxation-based decomposition, an estimation mechanism based on multiple-port network equivalent is adopted as the preconditioner to enhance the convergence of the proposed algorithm. The algorithm is presented in detail and validated both in terms of accuracy and capabilit

Electromechanical Quantum Simulators

Digital quantum simulators are among the most appealing applications of a quantum computer. Here we propose a universal, scalable, and integrated quantum computing platform based on tunable nonlinear electromechanical nano-oscillators. It is shown that very high operational fidelities for single and two qubits gates can be achieved in a minimal architecture, where qubits are encoded in the anharmonic vibrational modes of mechanical nanoresonators, whose effective coupling is mediated by virtual fluctuations of an intermediate superconducting artificial atom. An effective scheme to induce large single-phonon nonlinearities in nano-electromechanical devices is explicitly discussed, thus opening the route to experimental investigation in this direction. Finally, we explicitly show the very high fidelities that can be reached for the digital quantum simulation of model Hamiltonians, by using realistic experimental parameters in state-of-the art devices, and considering the transverse field Ising model as a paradigmatic example.Comment: 14 pages, 8 figure

An Efficient Integrated Circuit Simulator And Time Domain Adjoint Sensitivity Analysis

In this paper, we revisit time-domain adjoint sensitivity with a circuit theoretic approach and an efficient solution is clearly stated in terms of device level. Key is the linearization of the energy storage elements (e.g., capacitance and inductance) and nonlinear memoryless elements (e.g., MOS, BJT DC characteristics) at each time step. Due to the finite precision of computation, numerical errors that accumulate across timesteps can arise in nonlinear elements