Dynamic behavior-based control and world-embedded knowledge for interactive artificial intelligence

Video game designers depend on artificial intelligence to drive player experience in modern games. Therefore it is critical that AI not only be fast and computation- ally inexpensive, but also easy to incorporate with the design process. We address the problem of building computationally inexpensive AI that eases the game de- sign process and provides strategic and tactical behavior comparable with current industry-standard techniques. Our central hypothesis is that behavior-based characters in games can exhibit effec- tive strategy and coordinate in teams through the use of knowledge embedded in the world and a new dynamic approach to behavior-based control that enables charac- ters to transfer behavioral knowledge. We use dynamic extensions for behavior-based subsumption and world-embedded knowledge to simplify and enhance game character intelligence. We find that the use of extended affordances to embed knowledge in the world can greatly reduce the effort required to build characters and AI engines while increasing the effectiveness of the behavior controllers. In addition, we find that the technique of multi-character affordances can provide a simple mechanism for enabling team coordination. We also show that reactive teaming, enabled by dynamic extensions to the subsumption architecture, is effective in creating large adaptable teams of characters. Finally, we show that the command policy for reactive teaming can be used to improve performance of reactive teams for tactical situations

Innate-Values-driven Reinforcement Learning for Cooperative Multi-Agent Systems

Innate values describe agents' intrinsic motivations, which reflect their inherent interests and preferences to pursue goals and drive them to develop diverse skills satisfying their various needs. The essence of reinforcement learning (RL) is learning from interaction based on reward-driven (such as utilities) behaviors, much like natural agents. It is an excellent model to describe the innate-values-driven (IV) behaviors of AI agents. Especially in multi-agent systems (MAS), building the awareness of AI agents to balance the group utilities and system costs and satisfy group members' needs in their cooperation is a crucial problem for individuals learning to support their community and integrate human society in the long term. This paper proposes a hierarchical compound intrinsic value reinforcement learning model -- innate-values-driven reinforcement learning termed IVRL to describe the complex behaviors of multi-agent interaction in their cooperation. We implement the IVRL architecture in the StarCraft Multi-Agent Challenge (SMAC) environment and compare the cooperative performance within three characteristics of innate value agents (Coward, Neutral, and Reckless) through three benchmark multi-agent RL algorithms: QMIX, IQL, and QTRAN. The results demonstrate that by organizing individual various needs rationally, the group can achieve better performance with lower costs effectively.Comment: This paper was accepted by the 38th AAAI 2024 workshop: "Cooperative Multi-Agent Systems Decision-Making and Learning: From Individual Needs to Swarm Intelligence

Intelligent Operation System for the Autonomous Vehicle Fleet

Modular vehicles are vehicles with interchangeable substantial components also known as modules. Fleet modularity provides extra operational flexibility through on-field actions, in terms of vehicle assembly, disassembly, and reconfiguration (ADR). The ease of assembly and disassembly of modular vehicles enables them to achieve real-time fleet reconfiguration, which is proven as beneficial in promoting fleet adaptability and in saving ownership costs. The objective of military fleet operation is to satisfy uncertain demands on time while providing vehicle maintenance. To quantify the benefits and burdens from modularity in military operation, a decision support system is required to yield autonomously operation strategies for comparing the (near) optimal fleet performance for different vehicle architectures under diverse scenarios. The problem is challenging because: 1) fleet operation strategies are numerous, especially when modularity is considered; 2) operation actions are time-delayed and time-varying; 3) vehicle damages and demands are highly uncertain; 4) available capacity for ADR actions and vehicle repair is constrained. Finally, to explore advanced tactics enabled by fleet modularity, the competition between human-like and adversarial forces is required, where each force is capable to autonomously perceive and analyze field information, learn enemy's behavior, forecast enemy's actions, and prepare an operation plan accordingly. Currently, methodologies developed specifically for fleet competition are only valid for single type of resources and simple operation rules, which are impossible to implement in modular fleet operation. This dissertation focuses on a new general methodology to yield decisions in operating a fleet of autonomous military vehicles/robots in both conventional and modular architectures. First, a stochastic state space model is created to represent the changes in fleet dynamics caused by operation actions. Then, a stochastic model predictive control is customized to manage the system dynamics, which is capable of real-time decision making. Including modularity increases the complexity of fleet operation problem, a novel intelligent agent based model is proposed to ensure the computational efficiency and also imitate the collaborative decisions making process of human-like commanders. Operation decisions are distributed to several agents with distinct responsibility. Agents are designed in a specific way to collaboratively make and adjust decisions through selectively sharing information, reasoning the causality between events, and learning the other's behavior, which are achieved by real-time optimization and artificial intelligence techniques. To evaluate the impacts from fleet modularity, three operation problems are formulated: (i) simplified logistic mission scenario: operate a fleet to guarantee the readiness of vehicles at battlefields considering the stochasticity in inventory stocks and mission requirements; (ii) tactical mission scenario: deliver resources to battlefields with stochastic requirements of vehicle repairs and maintenance; (iii) attacker-defender game: satisfy the mission requirements with minimized losses caused by uncertain assaults from an enemy. The model is also implemented for a civilian application, namely the real-time management of reconfigurable manufacturing systems (RMSs). As the number of RMS configurations increases exponentially with the size of the line and demand changes frequently, two challenges emerge: how to efficiently select the optimal configuration given limited resources, and how to allocate resources among lines. According to the ideas in modular fleet operation, a new mathematical approach is presented for distributing the stochastic demands and exchanging machines or modules among lines (which are groups of machines) as a bidding process, and for adaptively configuring these lines and machines for the resulting shared demand under a limited inventory of configurable components.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/147588/1/lixingyu_2.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/147588/2/lixingyu_1.pd

Shared Autonomy via Hindsight Optimization

In shared autonomy, user input and robot autonomy are combined to control a robot to achieve a goal. Often, the robot does not know a priori which goal the user wants to achieve, and must both predict the user's intended goal, and assist in achieving that goal. We formulate the problem of shared autonomy as a Partially Observable Markov Decision Process with uncertainty over the user's goal. We utilize maximum entropy inverse optimal control to estimate a distribution over the user's goal based on the history of inputs. Ideally, the robot assists the user by solving for an action which minimizes the expected cost-to-go for the (unknown) goal. As solving the POMDP to select the optimal action is intractable, we use hindsight optimization to approximate the solution. In a user study, we compare our method to a standard predict-then-blend approach. We find that our method enables users to accomplish tasks more quickly while utilizing less input. However, when asked to rate each system, users were mixed in their assessment, citing a tradeoff between maintaining control authority and accomplishing tasks quickly

A Survey of Multi-Agent Human-Robot Interaction Systems

This article presents a survey of literature in the area of Human-Robot Interaction (HRI), specifically on systems containing more than two agents (i.e., having multiple humans and/or multiple robots). We identify three core aspects of ``Multi-agent" HRI systems that are useful for understanding how these systems differ from dyadic systems and from one another. These are the Team structure, Interaction style among agents, and the system's Computational characteristics. Under these core aspects, we present five attributes of HRI systems, namely Team size, Team composition, Interaction model, Communication modalities, and Robot control. These attributes are used to characterize and distinguish one system from another. We populate resulting categories with examples from recent literature along with a brief discussion of their applications and analyze how these attributes differ from the case of dyadic human-robot systems. We summarize key observations from the current literature, and identify challenges and promising areas for future research in this domain. In order to realize the vision of robots being part of the society and interacting seamlessly with humans, there is a need to expand research on multi-human -- multi-robot systems. Not only do these systems require coordination among several agents, they also involve multi-agent and indirect interactions which are absent from dyadic HRI systems. Adding multiple agents in HRI systems requires advanced interaction schemes, behavior understanding and control methods to allow natural interactions among humans and robots. In addition, research on human behavioral understanding in mixed human-robot teams also requires more attention. This will help formulate and implement effective robot control policies in HRI systems with large numbers of heterogeneous robots and humans; a team composition reflecting many real-world scenarios.Comment: 23 pages, 7 figure

Virtual Reality-Based Interface for Advanced Assisted Mobile Robot Teleoperation

[EN] This work proposes a new interface for the teleoperation of mobile robots based on virtual reality that allows a natural and intuitive interaction and cooperation between the human and the robot, which is useful for many situations, such as inspection tasks, the mapping of complex environments, etc. Contrary to previous works, the proposed interface does not seek the realism of the virtual environment but provides all the minimum necessary elements that allow the user to carry out the teleoperation task in a more natural and intuitive way. The teleoperation is carried out in such a way that the human user and the mobile robot cooperate in a synergistic way to properly accomplish the task: the user guides the robot through the environment in order to benefit from the intelligence and adaptability of the human, whereas the robot is able to automatically avoid collisions with the objects in the environment in order to benefit from its fast response. The latter is carried out using the well-known potential field-based navigation method. The efficacy of the proposed method is demonstrated through experimentation with the Turtlebot3 Burger mobile robot in both simulation and real-world scenarios. In addition, usability and presence questionnaires were also conducted with users of different ages and backgrounds to demonstrate the benefits of the proposed approach. In particular, the results of these questionnaires show that the proposed virtual reality based interface is intuitive, ergonomic and easy to use.This research was funded by the Spanish Government (Grant PID2020-117421RB-C21 funded byMCIN/AEI/10.13039/501100011033) and by the Generalitat Valenciana (Grant GV/2021/181).Solanes, JE.; Muñoz García, A.; Gracia Calandin, LI.; Tornero Montserrat, J. (2022). Virtual Reality-Based Interface for Advanced Assisted Mobile Robot Teleoperation. Applied Sciences. 12(12):1-22. https://doi.org/10.3390/app12126071122121

3D visual simulation of individual and crowd behavior in earthquake evacuation

Simulation of behaviors in emergencies is an interesting subject that helps to understand evacuation processes and to give out warnings for contingency plans. Individual and crowd behaviors in the earthquake are different from those under normal circumstances. Panic will spread in the crowd and cause chaos. Without considering emotion, most existing behavioral simulation methods analyze the movement of people from the point of view of mechanics. After summarizing existing studies, a new simulation method is discussed in this paper. First, 3D virtual scenes are constructed with the proposed platform. Second, an individual cognitive architecture, which integrates perception, motivation, behavior, emotion, and personality, is proposed. Typical behaviors are analyzed and individual evacuation animations are realized with data captured by motion capture devices. Quantitative descriptions are presented to describe emotional changes in individual evacuation. Facial expression animation is used to represent individuals’ emotions. Finally, a crowd behavior model is designed on the basis of a social force model. Experiments are carried out to validate the proposed method. Results showed that individuals’ behavior, emotional changes, and crowd aggregation can be well simulated. Users can learn evacuation processes from many angles. The method can be an intuitional approach to safety education and crowd management