FUNCTION AND DISSIPATION IN FINITE STATE AUTOMATA - FROM COMPUTING TO INTELLIGENCE AND BACK

Society has benefited from the technological revolution and the tremendous growth in computing powered by Moore\u27s law. However, we are fast approaching the ultimate physical limits in terms of both device sizes and the associated energy dissipation. It is important to characterize these limits in a physically grounded and implementation-agnostic manner, in order to capture the fundamental energy dissipation costs associated with performing computing operations with classical information in nano-scale quantum systems. It is also necessary to identify and understand the effect of quantum in-distinguishability, noise, and device variability on these dissipation limits. Identifying these parameters is crucial to designing more energy efficient computing systems moving forward. In this dissertation, we will provide a physical description of finite state automaton, an abstract tool commonly used to describe computational operations under the Referential Approach to physical information theory. We will derive the fundamental limits of dissipation associated with a state transition in deterministic and probabilistic finite state automaton, and propose efficacy measures to capture how well a particular state transition has been physically realized. We will use these dissipation bounds to understand the limits of dissipation during learning during training and testing phases in feed-forward and recurrent neural networks. This study of dissipation in neural network provides key hints at how dissipation is fundamentally intertwined with learning in physical systems. These ideas connecting energy dissipation, entropy and physical information provide the perfect toolkit to critically analyze the very foundations of computing, and our computational approaches to artificial intelligence. In the second part of this dissertation, we derive the non-equilibrium reliable low dissipation condition for predictive inference in self-organized systems. This brings together the central ideas of homeostasis, prediction and energy efficiency under a single non-equilibrium constraint. The work was further extended to study the relationship between adaptive learning and the reliable high dissipation conditions, and the exploitation-exploration trade-offs in active agents. Using these results, we will discuss the differences between observer dependent and independent computing, and propose an alternative novel descriptive framework of intelligence in physical systems using thermodynamics. This framework is called thermodynamic intelligence and will be used to guide the engineering methodologies (devices and architectures) required to implement these descriptions

Energy challenges for ICT

The energy consumption from the expanding use of information and communications technology (ICT) is unsustainable with present drivers, and it will impact heavily on the future climate change. However, ICT devices have the potential to contribute signi - cantly to the reduction of CO2 emission and enhance resource e ciency in other sectors, e.g., transportation (through intelligent transportation and advanced driver assistance systems and self-driving vehicles), heating (through smart building control), and manu- facturing (through digital automation based on smart autonomous sensors). To address the energy sustainability of ICT and capture the full potential of ICT in resource e - ciency, a multidisciplinary ICT-energy community needs to be brought together cover- ing devices, microarchitectures, ultra large-scale integration (ULSI), high-performance computing (HPC), energy harvesting, energy storage, system design, embedded sys- tems, e cient electronics, static analysis, and computation. In this chapter, we introduce challenges and opportunities in this emerging eld and a common framework to strive towards energy-sustainable ICT

Energy Requirement of Control: Comments on Szilard's Engine and Maxwell's Demon

In mathematical physical analyses of Szilard's engine and Maxwell's demon, a general assumption (explicit or implicit) is that one can neglect the energy needed for relocating the piston in Szilard's engine and for driving the trap door in Maxwell's demon. If this basic assumption is wrong, then the conclusions of a vast literature on the implications of the Second Law of Thermodynamics and of Landauer's erasure theorem are incorrect too. Our analyses of the fundamental information physical aspects of various type of control within Szilard's engine and Maxwell's demon indicate that the entropy production due to the necessary generation of information yield much greater energy dissipation than the energy Szilard's engine is able to produce even if all sources of dissipation in the rest of these demons (due to measurement, decision, memory, etc) are neglected.Comment: New, simpler and more fundamental approach utilizing the physical meaning of control-information and the related entropy production. Criticism of recent experiments adde

Ultimate Intelligence Part I: Physical Completeness and Objectivity of Induction

We propose that Solomonoff induction is complete in the physical sense via several strong physical arguments. We also argue that Solomonoff induction is fully applicable to quantum mechanics. We show how to choose an objective reference machine for universal induction by defining a physical message complexity and physical message probability, and argue that this choice dissolves some well-known objections to universal induction. We also introduce many more variants of physical message complexity based on energy and action, and discuss the ramifications of our proposals.Comment: Under review at AGI-2015 conference. An early draft was submitted to ALT-2014. This paper is now being split into two papers, one philosophical, and one more technical. We intend that all installments of the paper series will be on the arxi

Probabilistic metrology or how some measurement outcomes render ultra-precise estimates

We show on theoretical grounds that, even in the presence of noise, probabilistic measurement strategies (which have a certain probability of failure or abstention) can provide, upon a heralded successful outcome, estimates with a precision that exceeds the deterministic bounds for the average precision. This establishes a new ultimate bound on the phase estimation precision of particular measurement outcomes (or sequence of outcomes). For probe systems subject to local dephasing, we quantify such precision limit as a function of the probability of failure that can be tolerated. Our results show that the possibility of abstaining can set back the detrimental effects of noise.Comment: Improved version of arXiv:1407.6910 with an extended introduction where we clarify our approach to metrology, and probabilistic metrology in particular. Changed titl