Temporal Planning for Compilation of Quantum Approximate Optimization Algorithm Circuits

We investigate the application of temporal planners to the problem of compiling quantum circuits to newly emerging quantum hardware. While our approach is general, we focus our initial experiments on Quantum Approximate Optimization Algorithm (QAOA) circuits that have few ordering constraints and allow highly parallel plans. We report on experiments using several temporal planners to compile circuits of various sizes to a realistic hardware. This early empirical evaluation suggests that temporal planning is a viable approach to quantum circuit compilation

Specifying Meta-Level Architectures for Rule-Based Systems

Explicit and declarative representation of control knowledge and well-structured knowledge bases are crucial requirements for efficient development and maintenance of rule-based systems. The CATWEAZLE rule interpreter allows knowledge engineers to meet these requirements by partitioning rule bases and specifying meta-level architectures for control. Among others the following problems arise when providing tools for specifying meta-level architectures for control: 1. What is a suitable language to specify meta-level architectures for control? 2. How can a general and declarative language for meta-level architectures be efficiently interpreted? The thesis outlines solutions to both research questions provided by the CATWEAZLE rule interpreter: 1. CATWEAZLE provides a small set of concepts based on a separation of control knowledge in control strategies and control tactics and a further categorization of control strategies. 2. For rule-based systems it is efficient to extend the RETE algorithm such that control knowledge can be processed, too

Extending classical planning with state constraints: Heuristics and search for optimal planning

We present a principled way of extending a classical AI planning formalism with systems of state constraints, which relate - sometimes determine - the values of variables in each state traversed by the plan. This extension occupies an attractive middle ground between expressivity and complexity. It enables modelling a new range of problems, as well as formulating more efficient models of classical planning problems. An example of the former is planning-based control of networked physical systems - power networks, for example - in which a local, discrete control action can have global effects on continuous quantities, such as altering flows across the entire network. At the same time, our extension remains decidable as long as the satisfiability of sets of state constraints is decidable, including in the presence of numeric state variables, and we demonstrate that effective techniques for cost-optimal planning known in the classical setting - in particular, relaxation-based admissible heuristics - can be adapted to the extended formalism. In this paper, we apply our approach to constraints in the form of linear or non-linear equations over numeric state variables, but the approach is independent of the type of state constraints, as long as there exists a procedure that decides their consistency. The planner and the constraint solver interact through a well-defined, narrow interface, in which the solver requires no specialisation to the planning contextThis work was supported by ARC project DP140104219, “Robust AI Planning for Hybrid Systems”, and in part by ARO grant W911NF1210471 and ONR grant N000141210430

Learning the Structure of Continuous Markov Decision Processes

There is growing interest in artificial, intelligent agents which can operate autonomously for an extended period of time in complex environments and fulfill a variety of different tasks. Such agents will face different problems during their lifetime which may not be foreseeable at the time of their deployment. Thus, the capacity for lifelong learning of new behaviors is an essential prerequisite for this kind of agents as it enables them to deal with unforeseen situations. However, learning every complex behavior anew from scratch would be cumbersome for the agent. It is more plausible to consider behavior to be modular and let the agent acquire a set of reusable building blocks for behavior, the so-called skills. These skills might, once acquired, facilitate fast learning and adaptation of behavior to new situations. This work focuses on computational approaches for skill acquisition, namely which kind of skills shall be acquired and how to acquire them. The former is commonly denoted as skill discovery and the latter as skill learning . The main contribution of this thesis is a novel incremental skill acquisition approach which is suited for lifelong learning. In this approach, the agent learns incrementally a graph-based representation of a domain and exploits certain properties of this graph such as its bottlenecks for skill discovery. This thesis proposes a novel approach for learning a graph-based representation of continuous domains based on formalizing the problem as a probabilistic generative model. Furthermore, a new incremental agglomerative clustering approach for identifying bottlenecks of such graphs is presented. Thereupon, the thesis proposes a novel intrinsic motivation system which enables an agent to intelligently allocate time between skill discovery and skill learning in developmental settings, where the agent is not constrained by external tasks. The results of this thesis show that the resulting skill acquisition approach is suited for continuous domains and can deal with domain stochasticity and different explorative behavior of the agent. The acquired skills are reusable and versatile and can be used in multi-task and lifelong learning settings in high-dimensional problems

Quality Measures of Parameter Tuning for Aggregated Multi-Objective Temporal Planning

Parameter tuning is recognized today as a crucial ingredient when tackling an optimization problem. Several meta-optimization methods have been proposed to find the best parameter set for a given optimization algorithm and (set of) problem instances. When the objective of the optimization is some scalar quality of the solution given by the target algorithm, this quality is also used as the basis for the quality of parameter sets. But in the case of multi-objective optimization by aggregation, the set of solutions is given by several single-objective runs with different weights on the objectives, and it turns out that the hypervolume of the final population of each single-objective run might be a better indicator of the global performance of the aggregation method than the best fitness in its population. This paper discusses this issue on a case study in multi-objective temporal planning using the evolutionary planner DaE-YAHSP and the meta-optimizer ParamILS. The results clearly show how ParamILS makes a difference between both approaches, and demonstrate that indeed, in this context, using the hypervolume indicator as ParamILS target is the best choice. Other issues pertaining to parameter tuning in the proposed context are also discussed.Comment: arXiv admin note: substantial text overlap with arXiv:1305.116

A preliminary analysis of the Soar architecture as a basis for general intelligence

In this article we take a step towards providing an analysis of the Soar architecture as a basis for general intelligence. Included are discussions of the basic assumptions underlying the development of Soar, a description of Soar cast in terms of the theoretical idea of multiple levels of description, an example of Soar performing multi-column subtraction, and three analyses of Soar: its natural tasks, the sources of its power, and its scope and limitsPeer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/29595/1/0000684.pd

Ongelmien dekompositio konseptisuunnittelussa

Design embraces several disciplines dedicated to the production of artifacts and services. These disciplines are quite independent and only recently has psychological interest focused on them. Nowadays, the psychological theories of design, also called design cognition literature, describe the design process from the information processing viewpoint. These models co-exist with the normative standards of how designs should be crafted. In many places there are concrete discrepancies between these two in a way that resembles the differences between the actual and ideal decision-making. This study aimed to explore the possible difference related to problem decomposition. Decomposition is a standard component of human problem-solving models and is also included in the normative models of design. The idea of decomposition is to focus on a single aspect of the problem at a time. Despite its significance, the nature of decomposition in conceptual design is poorly understood and has only been preliminary investigated. This study addressed the status of decomposition in conceptual design of products using protocol analysis. Previous empirical investigations have argued that there are implicit and explicit decomposition, but have not provided a theoretical basis for these two. Therefore, the current research began by reviewing the problem solving and design literature and then composing a cognitive model of the solution search of conceptual design. The result is a synthetic view which describes recognition and decomposition as the basic schemata for conceptual design. A psychological experiment was conducted to explore decomposition. In the test, sixteen (N=16) senior students of mechanical engineering created concepts for two alternative tasks. The concurrent think-aloud method and protocol analysis were used to study decomposition. The results showed that despite the emphasis on decomposition in the formal education, only few designers (N=3) used decomposition explicitly and spontaneously in the presented tasks, although the designers in general applied a top-down control strategy. Instead, inferring from the use of structured strategies, the designers always relied on implicit decomposition. These results confirm the initial observations found in the literature, but they also suggest that decomposition should be investigated further. In the future, the benefits and possibilities of explicit decomposition should be considered along with the cognitive mechanisms behind decomposition. After that, the current results could be reinterpreted.Tuotteita ja palveluita suunnittelevia aloja on nykyään lukuisia. Suunnittelun eri alueet ovat kehittyneet itsenäisesti, ja viime aikoina niistä on kiinnostuttu myös psykologiassa. Suunnittelukognitioksi kutsutaan tutkimussuuntaa, jossa suunnittelua tutkitaan kognitiivisen psykologian näkökulmasta. Suunnittelukognition tulokset ovat suunnittelun normatiivisten mallien ohella kiinteä osa alan tietämystä. Kaikilta osin psykologinen todellisuus ei kuitenkaan vastaa mallien olettamuksia, vaan ne eroavat toisistaan kuten optimaalinen ja tosiasiallisen päätöksenteko. Tämä tutkimus selvitti mahdollista eroavaisuutta ongelmien dekomposition suhteen. Ongelmien dekompositio on menetelmä, joka sisältyy niin suunnittelun oppikirjoihin kuin ihmisen ongelmanratkaisun yleisiin malleihin. Dekomposition idea on helpottaa ongelmanratkaisua keskittämällä ongelmanratkaisijan huomio kerrallaan vain yhteen osaan ongelmasta. Dekomposition keskeisyydestä huolimatta dekomposition merkitys konseptisuunnittelussa on epäselvä ja niukasti tutkittu asia. Tässä työssä käytettiin protokolla-analyysia dekomposition tutkimiseksi. Aiemmissa kokeellisissa tutkimuksissa on esitetty dekomposition toimivan sekä implisiittisesti että eksplisiittisesti, mutta asiaa ei ole teoreettisesti perusteltu. Niinpä tässä tutkimuksessa käytiin ensin läpi viimeaikaisia suunnittelu- ja ongelmanratkaisuteorioita, joista koottiin kognitiivinen malli konseptisuunnittelun etsintävaiheesta. Tässä mallissa konseptisuunnittelun etsintävaihe nähdään tunnistusja dekompositio-skeemojen ohjaamana ongelmanratkaisuna. Teoreettista vaihetta seurasi empiirinen koe dekomposition tutkimiseksi. Kokeeseen osallistuneet kuusitoista (N=16) kokenutta koneensuunnittelun opiskelijaa tuottivat konsepteja kahdesta vaihtoehtoisesta aiheesta. Yhtäaikaista ääneenajattelua ja protokolla-analyysia hyödynnettiin dekomposition tutkimiseksi. Tulokset osoittivat, että huolimatta dekomposition korostamisesta koulutuksessa, ainoastaan muutama (N=3) suunnittelija käytti menetelmää spontaanisti ja eksplisiittisesti esitetyn kaltaisissa tehtävissä, vaikka tehtävät muuten ratkaistiin järjestelmällisesti käyttäen ylhäältä-alas-ohjausstrategiaa. Eksplisiittisen dekomposition käyttäminen tutkituissa tapauksissa ei myöskään tuottanut odotettuja tuloksia Eksplisiittisen dekomposition sijaan suunnittelijat käyttivät ohjausstrategioista päätellen implisiittistä dekompositiota. Nämä tulokset tukevat aiemmin tehtyjä havaintoja, mutta myös korostavat dekomposition lisätutkimuksen tarvetta. Tulevaisuudessa pitäisi selvittää, voidaanko dekomposition käyttöä tehostaa, millainen kognitiivinen prosessi dekompositio on ja mitkä sen käytön vaikutukset työskentelylle ovat. Näiden kysymysten selvittämisen jälkeen voidaan nykyisiä tuloksia arvioida uudelleen

Learning and planning in videogames via task decomposition

Artificial intelligence (AI) methods have come a long way in tabletop games, with computer programs having now surpassed human experts in the challenging games of chess, Go and heads-up no-limit Texas hold'em. However, a significant simplifying factor in these games is that individual decisions have a relatively large impact on the state of the game. The real world, however, is granular. Human beings are continually presented with new information and are faced with making a multitude of tiny decisions every second. Viewed in these terms, feedback is often sparse, meaning that it only arrives after one has made a great number of decisions. Moreover, in many real-world problems there is a continuous range of actions to choose from, and attaining meaningful feedback from the environment often requires a strong degree of action coordination. Videogames, in which players must likewise contend with granular time scales and continuous action spaces, are in this sense a better proxy for real-world problems, and have thus become regarded by many as the new frontier in games AI. Seemingly, the way in which human players approach granular decision-making in videogames is by decomposing complex tasks into high-level subproblems, thereby allowing them to focus on the "big picture". For example, in Super Mario World, human players seem to look ahead in extended steps, such as climbing a vine or jumping over a pit, rather than planning one frame at a time. Currently though, this type of reasoning does not come easily to machines, leaving many open research problems related to task decomposition. This thesis focuses on three such problems in particular: (1) The challenge of learning subgoals autonomously, so as to lessen the issue of sparse feedback. (2) The challenge of combining discrete planning techniques with extended actions whose durations and effects on the environment are uncertain. (3) The questions of when and why it is beneficial to reason over high-level continuous control variables, such as the velocity of a player-controlled ship, rather than over the most low-level actions available. We address these problems via new algorithms and novel experimental design, demonstrating empirically that our algorithms are more efficient than strong baselines that do not leverage task decomposition, and yielding insight into the types of environment where task decomposition is likely to be beneficial

Hierarchical Reinforcement Learning in Behavior and the Brain

Dissertation presented to obtain the Ph.D degree in Biology, NeuroscienceReinforcement learning (RL) has provided key insights to the neurobiology of learning and decision making. The pivotal nding is that the phasic activity of dopaminergic cells in the ventral tegmental area during learning conforms to a reward prediction error (RPE), as speci ed in the temporal-di erence learning algorithm (TD). This has provided insights to conditioning, the distinction between habitual and goal-directed behavior, working memory, cognitive control and error monitoring. It has also advanced the understanding of cognitive de cits in Parkinson's disease, depression, ADHD and of personality traits such as impulsivity.(...