Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field

On the development of slime mould morphological, intracellular and heterotic computing devices

The use of live biological substrates in the fabrication of unconventional computing (UC) devices is steadily transcending the barriers between science fiction and reality, but efforts in this direction are impeded by ethical considerations, the field’s restrictively broad multidisciplinarity and our incomplete knowledge of fundamental biological processes. As such, very few functional prototypes of biological UC devices have been produced to date. This thesis aims to demonstrate the computational polymorphism and polyfunctionality of a chosen biological substrate — slime mould Physarum polycephalum, an arguably ‘simple’ single-celled organism — and how these properties can be harnessed to create laboratory experimental prototypes of functionally-useful biological UC prototypes. Computing devices utilising live slime mould as their key constituent element can be developed into a) heterotic, or hybrid devices, which are based on electrical recognition of slime mould behaviour via machine-organism interfaces, b) whole-organism-scale morphological processors, whose output is the organism’s morphological adaptation to environmental stimuli (input) and c) intracellular processors wherein data are represented by energetic signalling events mediated by the cytoskeleton, a nano-scale protein network. It is demonstrated that each category of device is capable of implementing logic and furthermore, specific applications for each class may be engineered, such as image processing applications for morphological processors and biosensors in the case of heterotic devices. The results presented are supported by a range of computer modelling experiments using cellular automata and multi-agent modelling. We conclude that P. polycephalum is a polymorphic UC substrate insofar as it can process multimodal sensory input and polyfunctional in its demonstrable ability to undertake a variety of computing problems. Furthermore, our results are highly applicable to the study of other living UC substrates and will inform future work in UC, biosensing, and biomedicine

Statistical Physics of Vehicular Traffic and Some Related Systems

In the so-called "microscopic" models of vehicular traffic, attention is paid explicitly to each individual vehicle each of which is represented by a "particle"; the nature of the "interactions" among these particles is determined by the way the vehicles influence each others' movement. Therefore, vehicular traffic, modeled as a system of interacting "particles" driven far from equilibrium, offers the possibility to study various fundamental aspects of truly nonequilibrium systems which are of current interest in statistical physics. Analytical as well as numerical techniques of statistical physics are being used to study these models to understand rich variety of physical phenomena exhibited by vehicular traffic. Some of these phenomena, observed in vehicular traffic under different circumstances, include transitions from one dynamical phase to another, criticality and self-organized criticality, metastability and hysteresis, phase-segregation, etc. In this critical review, written from the perspective of statistical physics, we explain the guiding principles behind all the main theoretical approaches. But we present detailed discussions on the results obtained mainly from the so-called "particle-hopping" models, particularly emphasizing those which have been formulated in recent years using the language of cellular automata.Comment: 170 pages, Latex, figures include

【研究分野別】シーズ集 [英語版]

[英語版

Negative permittivity and permeability of gold nanorods metamaterials in UV- Vis region

In this article, we report the growth of gold nanorods on glass substrates and copper nanoparticle thin films by cylindrical direct current magnetron sputtering (CDCMS) at room temperature. The grown gold nanorods have short lengths of < 20nm and show negative optical parameters in UV-Vis region. So far negative permittivity and permeability were only shown for complex artificial structures. In a case of simple structures like gold nanorods, the negative optical parameters were only predicted by simulation methods and considering ideal structures and they were not yet reported by experimental groups, who has grown or synthesis gold nanorods by physical or chemical methods. The small size of gold nanorods and thickness of our samples compare to other experimental groups could be the reason of negative permittivity and permeability in our case. Low loss metamaterials with simultaneously negative permittivity and permeability are desired for practical applications in many optical devices such as optical switching, waveguides, modulators, and plasmonic antenna arrays. The optical properties of the grown gold nanorods were defined by ultraviolet- visible (UV-Vis) spectroscopy and their quality was assessed through multi-technique characterization using transmission electron microscopy (TEM), field emission scanning electron microscopy (FE-SEM), X-ray diffraction (XRD), and energy dispersed X-ray (EDX).Comment: 13 pages, 9 figures, 2 table