A Review of Symbolic, Subsymbolic and Hybrid Methods for Sequential Decision Making

The field of Sequential Decision Making (SDM) provides tools for solving Sequential Decision Processes (SDPs), where an agent must make a series of decisions in order to complete a task or achieve a goal. Historically, two competing SDM paradigms have view for supremacy. Automated Planning (AP) proposes to solve SDPs by performing a reasoning process over a model of the world, often represented symbolically. Conversely, Reinforcement Learning (RL) proposes to learn the solution of the SDP from data, without a world model, and represent the learned knowledge subsymbolically. In the spirit of reconciliation, we provide a review of symbolic, subsymbolic and hybrid methods for SDM. We cover both methods for solving SDPs (e.g., AP, RL and techniques that learn to plan) and for learning aspects of their structure (e.g., world models, state invariants and landmarks). To the best of our knowledge, no other review in the field provides the same scope. As an additional contribution, we discuss what properties an ideal method for SDM should exhibit and argue that neurosymbolic AI is the current approach which most closely resembles this ideal method. Finally, we outline several proposals to advance the field of SDM via the integration of symbolic and subsymbolic AI

Architecture for Reasoning in Hybrid Discrete-Continuous Domains

Hybrid domains are those featuring a mix of discrete and continuous variables. Recent research has resulted in sophisticated general purpose languages for modeling hybrid domains such as PDDL+ and H as well as efficient planning algorithms based on translation to logical formalisms. However, other reasoning tasks, such as execution monitoring and diagnosis, have not received as much attention. In this thesis, we address this shortcoming and propose execution monitoring and diagnostic reasoning algorithms based on action language H together with an agent architecture that combines planning, diagnostics, and execution monitoring for hybrid domains. The algorithms are based on an expanded translation of action language H to Constraint Answer Set Programming (CASP), which we developed for this project. We demonstrate our approach on two simple, but non-trivial scenarios including one that we tested on an actual robot.M.S.S.E., Software Engineering -- Drexel University, 201

Goal Reasoning: Papers from the ACS workshop

This technical report contains the 11 accepted papers presented at the Workshop on Goal Reasoning, which was held as part of the 2013 Conference on Advances in Cognitive Systems (ACS-13) in Baltimore, Maryland on 14 December 2013. This is the third in a series of workshops related to this topic, the first of which was the AAAI-10 Workshop on Goal-Directed Autonomy while the second was the Self-Motivated Agents (SeMoA) Workshop, held at Lehigh University in November 2012. Our objective for holding this meeting was to encourage researchers to share information on the study, development, integration, evaluation, and application of techniques related to goal reasoning, which concerns the ability of an intelligent agent to reason about, formulate, select, and manage its goals/objectives. Goal reasoning differs from frameworks in which agents are told what goals to achieve, and possibly how goals can be decomposed into subgoals, but not how to dynamically and autonomously decide what goals they should pursue. This constraint can be limiting for agents that solve tasks in complex environments when it is not feasible to manually engineer/encode complete knowledge of what goal(s) should be pursued for every conceivable state. Yet, in such environments, states can be reached in which actions can fail, opportunities can arise, and events can otherwise take place that strongly motivate changing the goal(s) that the agent is currently trying to achieve. This topic is not new; researchers in several areas have studied goal reasoning (e.g., in the context of cognitive architectures, automated planning, game AI, and robotics). However, it has infrequently been the focus of intensive study, and (to our knowledge) no other series of meetings has focused specifically on goal reasoning. As shown in these papers, providing an agent with the ability to reason about its goals can increase performance measures for some tasks. Recent advances in hardware and software platforms (involving the availability of interesting/complex simulators or databases) have increasingly permitted the application of intelligent agents to tasks that involve partially observable and dynamically-updated states (e.g., due to unpredictable exogenous events), stochastic actions, multiple (cooperating, neutral, or adversarial) agents, and other complexities. Thus, this is an appropriate time to foster dialogue among researchers with interests in goal reasoning. Research on goal reasoning is still in its early stages; no mature application of it yet exists (e.g., for controlling autonomous unmanned vehicles or in a deployed decision aid). However, it appears to have a bright future. For example, leaders in the automated planning community have specifically acknowledged that goal reasoning has a prominent role among intelligent agents that act on their own plans, and it is gathering increasing attention from roboticists and cognitive systems researchers. In addition to a survey, the papers in this workshop relate to, among other topics, cognitive architectures and models, environment modeling, game AI, machine learning, meta-reasoning, planning, selfmotivated systems, simulation, and vehicle control. The authors discuss a wide range of issues pertaining to goal reasoning, including representations and reasoning methods for dynamically revising goal priorities. We hope that readers will find that this theme for enhancing agent autonomy to be appealing and relevant to their own interests, and that these papers will spur further investigations on this important yet (mostly) understudied topic

Штучний інтелект на основі нейронної мережі для гри в жанрі стратегія в реальному часі

Магістерська дисертація міститься на 118 сторінках та включає 44 рисунки, 5 таблицю та 31 бібліографічні посилання. Вона складається з наступних розділів: вступ, 5 розділів для основної частини, висновки, перелік посилань та 8 додатків. Ключові слова: нейронні мережі, штучний інтелект, ієрархічна мережа задач, стратегії в реальному часі, розподілена система навчання нейронних мереж. Магістерська дисертація присвячена розробці та опису штучного інтелекту з використанням нейронних мереж для гри жанрі стратегія в реальному часі. Актуальність обраної теми полягає в підвищенні ефективності агентів штучного інтелекту для ігор в жанрі стратегій в реальному часі, а також використання розподіленої системи з централізованим сервером як елементу покращення ефективності модулів штучного інтелекту. Метою роботи є створення системи агентів штучного інтелекту з використанням штучних мереж для забезпечення високої ефективності роботи ботів в іграх жанру стратегій в реальному часі. Об’єктом дослідження є штучний інтелект в іграх жанру стратегій в реальному часі, зокрема модулі мікроуправління (тактичний) та макроуправління (стратегічний), з використанням штучних нейронних мереж в комбінації з іншими підходами. Предметом дослідження є побудований з використанням штучних нейронних мереж штучний інтелект для гри в жанрі стратегії в реальному часі, а також розподілена система навчання штучного інтелекту з використанням централізованого серверу.The master's dissertation is on 118 pages and includes 44 figures, 5 tables and 31 bibliographic references. It consists of the following sections: introduction, 5 sections for the main part, conclusions, list of references and 8 appendices. The master's dissertation is devoted to the development and description of artificial intelligence using neural networks for the game genre of real-time strategy. The relevance of the chosen topic is to increase the efficiency of artificial intelligence agents for games in the genre of real-time strategy, as well as the use of a distributed system with a centralized server as an element of improving the efficiency of artificial intelligence modules. The aim of the work is to create a system of artificial intelligence agents using artificial networks to ensure high efficiency of bots in games genre strategy in real time. The object of research is artificial intelligence in real-time strategy games, including modules of microcontrol (tactical) and macrocontrol (strategic), using artificial neural networks in combination with other approaches. The subject of the study is artificial intelligence built using artificial neural networks to play in the genre of real-time strategy, as well as a distributed system of artificial intelligence training using a centralized server

Drive-Based Utility-Maximizing Computer Game Non-Player Characters

This research examines the emergence of the five-string fiddle in contemporary North American fiddle culture within the past ten years. By interacting with leading artistlevel practitioners, the research documents the evolution and impact of the instrument to date in exploring the possibilities the five-string fiddle presents for musical performance and innovation. North American vernacular music and, in particular, the contemporary fiddle playing landscape, exemplifies virtousic and innovative idiomatic technique and improvisation as central to an overarching musical explosion, evidenced in the music of many high level, multi-stylistic contemporary practitioners. Within contemporary American fiddle performance, it is compelling to observe how many of the most innovative and highly regarded players now perform on five-string fiddles. The research uses a qualitative research methodology, drawing on interviews conducted with seven leading American fiddle players, each of whom has adopted the five-string fiddle in their own musical practice. The participants represent a rich cross section of American fiddle culture. They emerged naturally during the course of the literature review, and in-depth listening research, as particularly relevant sample cases. All participants were identified as leading exponents of the diversities encompassed in American fiddle music, between them sharing extensive professional recording, performance and academic experience, and all playing on five-string instruments. The research is further illuminated through practice, reflecting on my own musical work in illustrating how I have personally adopted the five-string fiddle, drawing influence from the research in demonstrating some wider possibilities of the instrument. This enquiry is important as it addresses the lack of specific research to date regarding the five-string fiddle, despite the significanance it holds for some of American fiddle music\u27s leading exponents, and consequently, for fiddle music itself. Equally significant, is the role of the instrument in facilitating the performance of innovative extended instrumental techniques, in particular, the five-string fiddles association with the rhythmic/percussive \u27chop\u27 bow techniques, now, so conspicuous within contemporary groove-based American string music. ix The findings of this research established the definitive emergence of the five-string fiddle, and subscribe that the five-string has now become a widely accepted part of the mainstream instrumentation in American music. This understanding emerges clearly through the words and practice of the participants. From this perspective, the research identifies the musical reasons that inspire the instruments popularity and elaborates through practice, the musical possibilities it presents to others. behaviour selection systems that have been used successfully in industry. The evaluations show that UDGOAP can outperform these systems in both environments. Another novel contribution of this thesis is smart ambiance. Smart ambiance is an area of space in a virtual world that holds information about the context of that space and uses this information to have non-player characters inside the space select more contextually appropriate actions. Information about the context comes from events that took place inside the smart ambiance, objects inside the smart ambiance, and the location of the smart ambiance. Smart ambiance can be used with any cost based planner. This thesis demonstrates dierent aspects of smart ambiance by causing an industry standard action planner to select more contextually appropriate behaviours than it otherwise would have without the smart ambiance

Qualitative and Quantitative Solution Diversity in Heuristic-Search and Case-Based Planning

Planning is a branch of Artificial Intelligence (AI) concerned with projecting courses of actions for executing tasks and reaching goals. AI Planning helps increase the autonomy of artificially intelligent agents and decrease the cognitive load burdening human planners working in challenging domains, such as the Mars exploration projects. Approaches to AI planning include first-principles heuristic search planning and case-based planning. The former conducts a heuristic-guided search in the solution space, while the latter generates new solutions by adapting solutions to previously-solved problems.The ability to generate not just one solution, but a set of meaningfully diverse solutions to each planning problem helps cater to a wider variety of user preferences and needs (which it may be difficult or even unfeasible to acquire and/or represent in their entirety), produce viable alternative courses of action to fall back on in case of failure, counter varied threats in intrusion detection, render computer games more compelling, and provide representative samples of the vast search spaces of planning problems.This work describes a general framework for generating diverse sets of solutions (i.e. courses of action) to planning problems. The general diversity-aware planning algorithm consists of iteratively generating solutions using a composite candidate-solution evaluation criterion taking into account both how promising the candidate solutions appear in their own right and on how likely they are to increase the overall diversity of the final set of solutions. This estimate of diversity is based on distance metrics, i.e. measures of the dissimilarity between two solutions. Distance metrics can be quantitative or qualitative.Quantitative distance measures are domain-independent. They require minimum knowledge engineering, but may not reflect dissimilarities that are truly meaningful. Qualitative distance metrics are domain-specific and reflect, based on the domain knowledge encoded within them, the kind of meaningful dissimilarities that might be identified by a person familiar with the domain.Based on the general framework for diversity-aware planning, three domain-independent planning algorithms have been implemented and are described and evaluated herein. DivFF is a diverse heuristic search planner for deterministic planning domains (i.e. domains for which the assumption is made that any action can only have one possible outcome). DivCBP is a diverse case-based planner, also for deterministic planning domains. DivNDP is a heuristic search planner for nondeterministic planning domains (i.e. domains the descriptions of which include actions with multiple possible outcomes). The experimental evaluation of the three algorithms is conducted on a computer game domain, chosen for its challenging characteristics, which include nondeterminism and dynamism. The generated courses of action are run in the game in order to ascertain whether they affect the game environment in diverse ways. This constitutes the test of their genuine diversity, which cannot be evaluated accurately based solely on their low-level structure.It is shown that all proposed planning systems successfully generate sets of diverse solutions using varied criteria for assessing solution dissimilarity. Qualitatively-diverse solution sets are demonstrated to constantly produce more diverse effects in the game environment than quantitatively-diverse solution sets.A comparison between the two planning systems for deterministic domains, DivCBP and DivFF, reveals the former to be more successful at consistently generating diverse sets of solutions. The reasons for this are investigated, thus contributing to the literature of comparative studies of first-principles and case-based planning approaches. Finally, an application of diversity in planning is showcased: simulating personality-trait variation in computer game characters. Sets of diverse solutions to both deterministic and nondeterministic planning problems are shown to successfully create diverse character behavior in the evaluation environment

Self Monitoring Goal Driven Autonomy Agents

The growing abundance of autonomous systems is driving the need for robust performance. Most current systems are not fully autonomous and often fail when placed in real environments. Via self-monitoring, agents can identify when their own, or externally given, boundaries are violated, thereby increasing their performance and reliability. Specifically, self-monitoring is the identification of unexpected situations that either (1) prohibit the agent from reaching its goal(s) or (2) result in the agent acting outside of its boundaries. Increasingly complex and open environments warrant the use of such robust autonomy (e.g., self-driving cars, delivery drones, and all types of future digital and physical assistants). The techniques presented herein advance the current state of the art in self-monitoring, demonstrating improved performance in a variety of challenging domains. In the aforementioned domains, there is an inability to plan for all possible situations. In many cases all aspects of a domain are not known beforehand, and, even if they were, the cost of encoding them is high. Self-monitoring agents are able to identify and then respond to previously unexpected situations, or never-before-encountered situations. When dealing with unknown situations, one must start with what is expected behavior and use that to derive unexpected behavior. The representation of expectations will vary among domains; in a real-time strategy game like Starcraft, it could be logically inferred concepts; in a mars rover domain, it could be an accumulation of actions\u27 effects. Nonetheless, explicit expectations are necessary to identify the unexpected. This thesis lays the foundation for self-monitoring in goal driven autonomy agents in both rich and expressive domains and in partially observable domains. We introduce multiple techniques for handling such environments. We show how inferred expectations are needed to enable high level planning in real-time strategy games. We show how a hierarchical structure of Goal-driven Autonomy (GDA) enables agents to operate within large state spaces. Within Hierarchical Task Network planning, we show how informed expectations identify states that are likely to prevent an agent from reaching its goals in dynamic domains. Finally, we give a model of expectations for self-monitoring at the meta-cognitive level, and empirical results of agents equipped with and without metacognitive expectations

Intelligent adaptive underwater sensor networks

Autonomous Underwater Vehicle (AUV) technology has reached a sufficient maturity level to be considered a suitable alternative to conventional Mine Countermeasures (MCM). Advantages of using a network of AUVs include time and cost efficiency, no personnel in the minefield, and better data collection. A major limitation for underwater robotic networks is the poor communication channel. Currently, acoustics provides the only means to send messages beyond a few metres in shallow water, however the bandwidth and data rate are low, and there are disturbances, such as multipath and variable channel delays, making the communication non-reliable. The solution this thesis proposes using a network of AUVs for MCM is the Synchronous Rendezvous (SR) method --- dynamically scheduling meeting points during the mission so the vehicles can share data and adapt their future actions according to the newly acquired information. Bringing the vehicles together provides a robust way of exchanging messages, as well as means for regular system monitoring by an operator. The gains and losses of the SR approach are evaluated against a benchmark scenario of vehicles having their tasks fixed. The numerical simulation results show the advantage of the SR method in handling emerging workload by adaptively retasking vehicles. The SR method is then further extended into a non-myopic setting, where the vehicles can make a decision taking into account how the future goals will change, given the available resource and estimation of expected workload. Simulation results show that the SR setting provides a way to tackle the high computational complexity load, common for non-myopic solutions. Validation of the SR method is based on trial data and experiments performed using a robotics framework, MOOS-IvP. This thesis develops and evaluates the SR method, a mission planning approach for underwater robotic cooperation in communication and resource constraint environment

Foundations of Trusted Autonomy

Trusted Autonomy; Automation Technology; Autonomous Systems; Self-Governance; Trusted Autonomous Systems; Design of Algorithms and Methodologie