A Study of AI Population Dynamics with Million-agent Reinforcement Learning

We conduct an empirical study on discovering the ordered collective dynamics obtained by a population of intelligence agents, driven by million-agent reinforcement learning. Our intention is to put intelligent agents into a simulated natural context and verify if the principles developed in the real world could also be used in understanding an artificially-created intelligent population. To achieve this, we simulate a large-scale predator-prey world, where the laws of the world are designed by only the findings or logical equivalence that have been discovered in nature. We endow the agents with the intelligence based on deep reinforcement learning (DRL). In order to scale the population size up to millions agents, a large-scale DRL training platform with redesigned experience buffer is proposed. Our results show that the population dynamics of AI agents, driven only by each agent's individual self-interest, reveals an ordered pattern that is similar to the Lotka-Volterra model studied in population biology. We further discover the emergent behaviors of collective adaptations in studying how the agents' grouping behaviors will change with the environmental resources. Both of the two findings could be explained by the self-organization theory in nature.Comment: Full version of the paper presented at AAMAS 2018 (International Conference on Autonomous Agents and Multiagent Systems

Computing Truth of Logical Statements in Multi-Agents’ Environment

Thispaperdescribeslogical models and computational algorithmsforlogical statements(specs) including various versions ofChanceDiscovery(CD).The approachisbased attemporal multi-agentlogic. Prime question is how to express most essential properties of CD in terms of temporal logic (branching time multi-agents’ logic or a linear one), how to define CD by formulas in logical language. We, as an example, introduce several formulas in the language of temporal multi-agent logic which may express essential properties of CD. Then we study computational questions (in particular, using some light modification of the standard filtration technique we show that the constructed logic has the finite-model property with effectively computable upper bound; this proves that the logic is decidable and provides a decision algorithm). At the final part of the paper we consider interpretation of CD via uncertainty and plausibility in an extension ofthelineartemporallogicLTL and computationfortruth values(satisfiability) ofits formulas.Представленная статья посвящена построению логических моделей различных версий теории случайных открытий (СО) и описанию вычислительных алгоритмов для логических высказываний. Предлагаемый нами подход основывается на многоагентной временной логике. Главный вопрос состоит в том, как можно было бы выразить самые существенные свойства СО в терминах временной логики, многоагентной логики с ветвящимся временем или линейной логики и вообще как определить СО с помощью формул языка логики. Нами в статье введено несколько формул на языке многоагентной временной логики, которые способны выразить существенные свойства СО. Используя некоторую модифицированную стандартную технику фильтрации, мы показали, что сконструированная таким образом логика имеет свойство финитной аппроксимируемости с эффективно вычислимой верхней границей. Это доказывает, что такая логика разрешима и нами предъявлен алгоритм разрешения. В заключительной части статьи мы рассматриваем интерпретацию СО посредством неопределённости и вероятности в расширении временной линейной логики и вычисление истинностных значений её формул

Agent and cyber-physical system based self-organizing and self-adaptive intelligent shopfloor

The increasing demand of customized production results in huge challenges to the traditional manufacturing systems. In order to allocate resources timely according to the production requirements and to reduce disturbances, a framework for the future intelligent shopfloor is proposed in this paper. The framework consists of three primary models, namely the model of smart machine agent, the self-organizing model, and the self-adaptive model. A cyber-physical system for manufacturing shopfloor based on the multiagent technology is developed to realize the above-mentioned function models. Gray relational analysis and the hierarchy conflict resolution methods were applied to achieve the self-organizing and self-adaptive capabilities, thereby improving the reconfigurability and responsiveness of the shopfloor. A prototype system is developed, which has the adequate flexibility and robustness to configure resources and to deal with disturbances effectively. This research provides a feasible method for designing an autonomous factory with exception-handling capabilities

Learning and Discovery

We formulate a dynamic framework for an individual decision-maker within which discovery of previously unconsidered propositions is possible. Using a standard game-theoretic representation of the state space as a tree structure generated by the actions of agents (including acts of nature), we show how unawareness of propositions can be represented by a coarsening of the state space. Furthermore we develop a semantics rich enough to describe the individual's awareness that currently undiscovered propositions may be discovered in the future. Introducing probability concepts, we derive a representation of ambiguity in terms of multiple priors, reflecting implicit beliefs about undiscovered proposition, and derive conditions for the special case in which standard Bayesian learning may be applied to a subset of unambiguous propositions. Finally, we consider exploration strategies appropriate to the context of discovery, comparing and contrasting them with learning strategies appropriate to the context of justification, and sketch applications to scientific research and entrepreneurship.

Interaction and Experience in Enactive Intelligence and Humanoid Robotics

We overview how sensorimotor experience can be operationalized for interaction scenarios in which humanoid robots acquire skills and linguistic behaviours via enacting a “form-of-life”’ in interaction games (following Wittgenstein) with humans. The enactive paradigm is introduced which provides a powerful framework for the construction of complex adaptive systems, based on interaction, habit, and experience. Enactive cognitive architectures (following insights of Varela, Thompson and Rosch) that we have developed support social learning and robot ontogeny by harnessing information-theoretic methods and raw uninterpreted sensorimotor experience to scaffold the acquisition of behaviours. The success criterion here is validation by the robot engaging in ongoing human-robot interaction with naive participants who, over the course of iterated interactions, shape the robot’s behavioural and linguistic development. Engagement in such interaction exhibiting aspects of purposeful, habitual recurring structure evidences the developed capability of the humanoid to enact language and interaction games as a successful participant

Reinventing learning: a design-research odyssey

Design research is a broad, practice-based approach to investigating problems of education. This approach can catalyze the development of learning theory by fostering opportunities for transformational change in scholars’ interpretation of instructional interactions. Surveying a succession of design-research projects, I explain how challenges in understanding students’ behaviors promoted my own recapitulation of a historical evolution in educators’ conceptualizations of learning—Romantic, Progressivist, and Synthetic (Schön, Intuitive thinking? A metaphor underlying some ideas of educational reform (working paper 8). Division for Study and Research in Education, MIT, Cambridge, 1981)—and beyond to a proposed Systemic view. In reflection, I consider methodological adaptations to design-research practice that may enhance its contributions in accord with its objectives