Reinforcement Learning: A Survey

This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word ``reinforcement.'' The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state. It concludes with a survey of some implemented systems and an assessment of the practical utility of current methods for reinforcement learning.Comment: See http://www.jair.org/ for any accompanying file

Towards homeostatic architecture: simulation of the generative process of a termite mound construction

This report sets out to the theme of the generation of a ‘living’, homeostatic and self-organizing architectural structure. The main research question this project addresses is what innovative techniques of design, construction and materials could prospectively be developed and eventually applied to create and sustain human-made buildings which are mostly adaptive, self-controlled and self-functioning, without option to a vast supply of materials and peripheral services. The hypothesis is that through the implementation of the biological building behaviour of termites, in terms of collective construction mechanisms that are based on environmental stimuli, we could achieve a simulation of the generative process of their adaptive structures, capable to inform in many ways human construction. The essay explicates the development of the 3-dimensional, agent-based simulation of the termite collective construction and analyzes the results, which involve besides physical modelling of the evolved structures. It finally elucidates the potential of this emerging and adaptive architectural performance to be translated to human practice and thus enlighten new ecological engineering and design methodologies

A Qualitative Study of the Nascent Entrepreneurial Process of an Immigrant within an Irish Context of a Cross-Cultural Adaptation

This thesis intersects the theoretical fields of nascent entrepreneurship and cross-cultural adaptation. The key research question that guides this research is: What is the nascent entrepreneurial journey of an immigrant within an Irish context of cross cultural adaptation? The study explores the notion that the nascent entrepreneurial and cross-cultural adaptation journeys of immigrants in this study are intertwined and dynamically interact with each other. Despite much academic interest in nascent entrepreneurship, there is a lack of qualitative studies that focus on understanding the nascent entrepreneurial process of immigrants on a longitudinal basis. Additionally, the immigrant/ethnic specific entrepreneurship theories frequently overlook the nascent entrepreneurial experiences of an individual and instead focus on group level analysis with a limited application of cross-cultural adaptation variables. It is within this theoretical space that this research seeks to make its contribution. Understanding this under-researched phenomenon within an Irish context adds an additional research opportunity. Methodologically based on a qualitative, longitudinal quasi-ethnographic research approach, the study focused on a close and in-depth investigation of six immigrants over a period of 18 months. The researcher, who is also an immigrant, used the methods of participant observation, qualitative interviewing, and informal communication to create new insights and enlighten the nature of this elusive process. The findings of the research enhanced the understanding of the nascent entrepreneurial journey of an immigrant situated within an Irish context of cross-cultural adaptation. The study design gave voices to the individual immigrants which has been missing from previous studies. Capturing the arrival of an economic recession during the time of the study, the research showed how the individuals negotiated their positions and resources, and how they dynamically moved either closer or further away from achieving their aims. These aims, which were simple in theory, yet complicated in practice, were centred around the continuous efforts of moving from surviving towards living

Case studies for a new IoT programming paradigm: Fluidware

A number of scientific and technological advancements enabled turning the Internet of Things vision into reality. However, there is still a bottleneck in designing and developing IoT applications and services: each device has to be programmed individually, and services are deployed to specific devices. The Fluidware approach advocates that to truly scale and raise the level of abstraction a novel perspective is needed, focussing on device ensembles and dynamic allocation of resources. In this paper, we motivate the need for such a paradigm shift through three case studies emphasising a mismatch between state of art solutions and desired properties to achieve

Voluntary Agreements in Environmental Protection - Experiences in Germany and Future Perspectives

A trend towards softer regulation, especially in the form of negotiated environmental agreements, is observable in national and international environmental policies. Such agreements are controversial, because there are fears that government will relinquish its responsibility for environmental protection. This paper analyses recent experiences with voluntary agreements in Germany. Topical German examples that have prompted public debates include the takeback agreement for cars, the voluntary agreement made by a number of industries on a C02 reduction by the year 2005 and the voluntary agreement made by the automobile industry on the development of energy-efficient cars. Proponents of voluntary agreements argue that this instrument provides incentives to the business sector for the development of efficient, innovative and environmentally-friendly solutions. Analysing the examples mentioned above, we conclude that it is hard to detect solutions derserving such attributes. These agreements are unlikely to produce results that go beyond what industry would have done in any case and they avoid using economic incentives. The agreements are' non-binding and unenforceable, with the negotiating process leading to a watering down of the environmental goals government had originally aimed at. A preference for negotiated solutions on principle, as currently espoused by the Federal Government in Germany, seems to be counterproductive. If the government clearly signals its willingness to refrain from using regulatory or economic instruments in favour of industry agreements, it weakens its negotiating position. The government also limits its options should the implementation of the agreement prove unsatisfactory. Government needs to be in control in order to leave its choice of policy instruments open and to be flexible. In a last step, we derive some general conclusions concerning reasonable strategies and applications of voluntary agreements within the European Union. --negotiated agreements,climate protection,circular economy,economic instruments of environmental policy

Design and Development of the Reactive BGP peering in Software-Defined Routing Exchanges

The Software-Defined Networking (SDN) is considered to be an improved solution for applying flexible control and operation recently in the network. Its characteristics include centralized management, global view, as well as fast adjustment and adaptation. Many experimental and research networks have already migrated to the SDN-enabled architecture. As the global network continues to grow in a fast pace, how to use SDN to improve the networking fields becomes a popular topic in research. One of the interesting topics is to enable routing exchanges among the SDN-enabled network and production networks. However, considering that many production networks are still operated on legacy architecture, the enabled SDN routing functionalities have to support hybrid mode in operation. In this paper, we propose a routing exchange mechanism by enabling reactive BGP peering actions among the SDN and legacy network components. The results of experiments show that our SDN controller is able to mask as an Autonomous System (AS) to exchange routing information with other BGP routers