Generalized Off-Policy Actor-Critic

We propose a new objective, the counterfactual objective, unifying existing objectives for off-policy policy gradient algorithms in the continuing reinforcement learning (RL) setting. Compared to the commonly used excursion objective, which can be misleading about the performance of the target policy when deployed, our new objective better predicts such performance. We prove the Generalized Off-Policy Policy Gradient Theorem to compute the policy gradient of the counterfactual objective and use an emphatic approach to get an unbiased sample from this policy gradient, yielding the Generalized Off-Policy Actor-Critic (Geoff-PAC) algorithm. We demonstrate the merits of Geoff-PAC over existing algorithms in Mujoco robot simulation tasks, the first empirical success of emphatic algorithms in prevailing deep RL benchmarks.Comment: NeurIPS 201

Software agents in music and sound art research/creative work: Current state and a possible direction

Composers, musicians and computer scientists have begun to use software-based agents to create music and sound art in both linear and non-linear (non-predetermined form and/or content) idioms, with some robust approaches now drawing on various disciplines. This paper surveys recent work: agent technology is first introduced, a theoretical framework for its use in creating music/sound art works put forward, and an overview of common approaches then given. Identifying areas of neglect in recent research, a possible direction for further work is then briefly explored. Finally, a vision for a new hybrid model that integrates non-linear, generative, conversational and affective perspectives on interactivity is proposed

Introduction to the Special Issue: The AgentLink III Technical Forums

This article introduces the special issue of ACM Transactions on Autonomous and Adaptive Systems devoted to research papers arising from the three Technical Forum Group meetings held in 2004 and 2005 that were organized and sponsored by the European FP6 Coordination Action AgentLink III

CARMA : complete autonomous responsible management agent (system)

University of Technology, Sydney. Faculty of Engineering and Information Technology.The continuing expansion of telecommunication service domains, from Quality of Service guaranteed connectivity to ubiquitous cloud environments, has introduced an ever increasing level of complexity in the field of service management. This complexity arises not only from the sheer variability in service requirements but also through the required but ill-defined interaction of multiple organisations and providers. As a result of this complexity and variability, the provisioning and performance of current services is adversely affected, often with little or no accountability to the users of the service. This exposes a need for total coverage in the management of such complex services, a system which provides for service responsibility. Service responsibility is defined as the provisioning of service resilience and the judgement of service risk across all the service components. To be effective in responsible management for current complex services, any framework must be able to interact with multiple providers and management systems. The CARMA framework proposed by this thesis, aims to fulfil these requirements through a multi-agent system, that is based in a global market, and can negotiate and be responsible for multiple complex services. The research presented in this thesis draws upon previous research in the fields of Network Management and Cloud service management, and utilises agent technology to build a system that is capable of providing resilient and risk aware management of services comprised of multiple providers. To this end the research aims to present the architecture, agent functionality and interactions of the CARMA system, as well as the structure of the marketplace, contract specification and risk management. As the scope and concepts of the proposed system are relatively unexplored, a model and simulation were developed to verify the concepts, explore the issues, assess the assumptions and validate the system. The results of the simulation determined that the introduction of CARMA has the potential to reduce the risk in contracting new services, increase the reliability of contracted services, and increase the utility of providers participating in the market

Organization Development for Social Change

The field of organization development (OD) has emerged from efforts to improve the performance of organizations, largely in the for-profit sector but more recently in the public and not-for-profit sectors as well. This paper examines how OD concepts and tools can be used to solve problems and foster constructive change at the societal level as well. It examines four areas in which OD can make such contributions: (1) strengthening social change-focused organizations, (2) scaling up the impacts of such agencies, (3) creating new inter-organizational systems, and (4) changing contexts that shape the action of actors strategic to social change. It discusses examples and the kinds of change agent roles and interventions that are important for each. Finally, it discusses some implications for organization development intervention, practitioners, and the field at large.This publication is Hauser Center Working Paper No. 25. The Hauser Center Working Paper Series was launched during the summer of 2000. The Series enables the Hauser Center to share with a broad audience important works-in-progress written by Hauser Center scholars and researchers