Nonstrict hierarchical reinforcement learning for interactive systems and robots

Conversational systems and robots that use reinforcement learning for policy optimization in large domains often face the problem of limited scalability. This problem has been addressed either by using function approximation techniques that estimate the approximate true value function of a policy or by using a hierarchical decomposition of a learning task into subtasks. We present a novel approach for dialogue policy optimization that combines the benefits of both hierarchical control and function approximation and that allows flexible transitions between dialogue subtasks to give human users more control over the dialogue. To this end, each reinforcement learning agent in the hierarchy is extended with a subtask transition function and a dynamic state space to allow flexible switching between subdialogues. In addition, the subtask policies are represented with linear function approximation in order to generalize the decision making to situations unseen in training. Our proposed approach is evaluated in an interactive conversational robot that learns to play quiz games. Experimental results, using simulation and real users, provide evidence that our proposed approach can lead to more flexible (natural) interactions than strict hierarchical control and that it is preferred by human users

Multi Agent Systems in Logistics: A Literature and State-of-the-art Review

Based on a literature survey, we aim to answer our main question: “How should we plan and execute logistics in supply chains that aim to meet today’s requirements, and how can we support such planning and execution using IT?†Today’s requirements in supply chains include inter-organizational collaboration and more responsive and tailored supply to meet specific demand. Enterprise systems fall short in meeting these requirements The focus of planning and execution systems should move towards an inter-enterprise and event-driven mode. Inter-organizational systems may support planning going from supporting information exchange and henceforth enable synchronized planning within the organizations towards the capability to do network planning based on available information throughout the network. We provide a framework for planning systems, constituting a rich landscape of possible configurations, where the centralized and fully decentralized approaches are two extremes. We define and discuss agent based systems and in particular multi agent systems (MAS). We emphasize the issue of the role of MAS coordination architectures, and then explain that transportation is, next to production, an important domain in which MAS can and actually are applied. However, implementation is not widespread and some implementation issues are explored. In this manner, we conclude that planning problems in transportation have characteristics that comply with the specific capabilities of agent systems. In particular, these systems are capable to deal with inter-organizational and event-driven planning settings, hence meeting today’s requirements in supply chain planning and execution.supply chain;MAS;multi agent systems

Astrobiology and Society in Europe Today

This book describes the state of astrobiology in Europe today and its relation to the European society at large. With contributions from authors in more than 20 countries and over 30 scientific institutions worldwide, the document illustrates the societal implications of astrobiology and the positive contribution that astrobiology can make to European society. The book has two main objectives: 1. It recommends the establishment of a European Astrobiology Institute (EAI) as an answer to a series of challenges relating to astrobiology but also European research, education, and society at large. 2. It also acknowledges the societal implications of astrobiology, and thus the role of the social sciences and humanities in optimizing the positive contribution that astrobiology can make to the lives of the people of Europe and the challenges they face

Affordable Generative Agents

The emergence of large language models (LLMs) has significantly advanced the simulation of believable interactive agents. However, the substantial cost on maintaining the prolonged agent interactions poses challenge over the deployment of believable LLM-based agents. Therefore, in this paper, we develop Affordable Generative Agents (AGA), a framework for enabling the generation of believable and low-cost interactions on both agent-environment and inter-agents levels. Specifically, for agent-environment interactions, we substitute repetitive LLM inferences with learned policies; while for inter-agent interactions, we model the social relationships between agents and compress auxiliary dialogue information. Extensive experiments on multiple environments show the effectiveness and efficiency of our proposed framework. Also, we delve into the mechanisms of emergent believable behaviors lying in LLM agents, demonstrating that agents can only generate finite behaviors in fixed environments, based upon which, we understand ways to facilitate emergent interaction behaviors. Our code is publicly available at: \url{https://github.com/AffordableGenerativeAgents/Affordable-Generative-Agents}

The Systems Thinking Learning Lens: An Exploratory Study of Executives' Mental Models

It has become progressively difficult for businesses to tackle unanticipated events and define the influencers that generate unintended business consequences. As such, uncertain and ambiguous situations are now the prescriptive norm for many companies. Executives are at the forefront of having to make sense of the uncertainty to seek the ideal decision pathway. The purpose of this exploratory research study was to seek what is known about learning how to develop a systems thinking mental model by exploring the perceptions and narratives of 12 global executives working in the United Arab Emirates (UAE) within complex adaptive systems (CAS) and their understanding of their thinking patterns that may have assisted in learning how to develop a systems thinking mental model to manage business ambiguity. Three research questions were developed to identify the types of experiences, perceptions, thinking patterns, and enablers—be they within the individual, organizational, or environmental context—that may have provided a strategic learning path. The research questions include: (a) What characterizes the mental models the executives hold (the distinct nature or features of their beliefs, behaviors, and principles)?; (b) What are the experiences that provide the scaffolding in developing a systems thinking mental model (experiences and events)?; and (c) What aspects of the individual, organizational, and environmental interactions enable individuals to learn how to develop a systems thinking capacity (relationships, systems, and elements)? The qualitative exploratory research study used three data collection methods: (a) semi-structured interviews, (b) focus group session, and (c) demographic questionnaire. The researcher concluded from the findings, analysis, interpretations, and synthesis that: (a) a systems thinking mental model is reflective of and responsive to different elements, situations, and influencers; (b) certain behaviors are an integral part of a systems thinking mental model; (c) informal learning experiences in ambiguous and uncertain situations may provide an ambiguous thinking learning pathway; and (d) learning through social, cultural, and operational systems is an under-utilized strategic intent