Probabilistic Modeling for Game Content Creation and Adaption

Dynamic Difficulty Adjustment studies how games can adapt content totheir users’ skill level, aiming to keep them in flow. Most of these methodsmaximize engagement or minimize churn by adapting factors like the opponentAI or the availability of resources. However, such methods do notmaintain a model of the player, and use technologies that are highly specificto the games in which they are tested (e.g. requiring forward modelsfor enemy AIs based on planning agents). Designers may also intend tofind content that is more difficult/easier on purpose, and current methodsdo not allow for such targeting.This thesis proposes and tests a framework for adapting game content tousers based on Bayesian Optimization, giving designers flexibility whenchoosing which skill level to target. Starting with a design space, a metricto be measured, a prior over this metric, and a target value, our frameworkquickly searches possible levels/tasks for one with ideal difficulty (i.e. closeto the specified target). In the process, our framework maintains a simpledata-driven model of the player, which could be used for further decisionmakingand analysis.We test this framework in two settings: adapting content to planning agentsbased on search algorithms likeMonte Carlo Tree Search and Rolling HorizonEvolution in a dungeon crawler-type game, and adapting both Sudokupuzzles and dungeon crawler levels to players. Our framework successfullyadapts content to planning agents as long as their skill level is not extreme,and takes roughly 7 iterations to find an appropriate Sudoku puzzle.Additionally, instead of relying on designers to specify a real-valued encodingof the content (e.g. the number of pre-filled cells in a Sudoku puzzle),we investigate learning this encoding automatically usingDeep GenerativeModels. In other words, we explore design spaces learned as latent spacesof Variational Autoencoders using tile-based representations of games likeSuperMario Bros and The Legend of Zelda.Our final contribution is a novel way of interpolating, sampling and optimizingin the playable regions of latent spaces of Variational Autoencoders,and addresses the challenge that generative models are not always guaranteedto decode playable content. This contribution, based on differentialgeometry, is inspired by recent advancements in domains like robotics andproteinmodeling. We combine these ideas of safe generation with contentoptimization and propose a restricted version of Bayesian Optimization,which optimizes content inside playable regions. We see a clear trade-off:restricting the latent space to playable regions decreases the diversity ofthe generated content, as well as the quality of the optimal values in theoptimization.In summary, this thesis studies applications of Bayesian Optimization andDeep Generative Models to the problem of creating and adapting gamecontent to users. We develop a framework that quickly finds relevant levelsin settings varying from corpora of levels to the latent spaces of generativemodels, and we show in experiments involving both human and artificialplayers that this framework finds appropriate game content in a few iterations.This framework is readily applicable, and could be used to creategames that learn and adapt to their players.<br/

Learning Hierarchical Compositional Task Definitions through Online Situated Interactive Language Instruction

Artificial agents, from robots to personal assistants, have become competent workers in many settings and embodiments, but for the most part, they are limited to performing the capabilities and tasks with which they were initially programmed. Learning in these settings has predominately focused on learning to improve the agent’s performance on a task, and not on learning the actual definition of a task. The primary method for imbuing an agent with the task definition has been through programming by humans, who have detailed knowledge of the task, domain, and agent architecture. In contrast, humans quickly learn new tasks from scratch, often from instruction by another human. If we desire AI agents to be flexible and dynamically extendable, they will need to emulate these learning capabilities, and not be stuck with the limitation that task definitions must be acquired through programming. This dissertation explores the problem of how an Interactive Task Learning agent can learn the complete definition or formulation of novel tasks rapidly through online natural language instruction from a human instructor. Recent advances in natural language processing, memory systems, computer vision, spatial reasoning, robotics, and cognitive architectures make the time ripe to study how knowledge can be automatically acquired, represented, transferred, and operationalized. We present a learning approach embodied in an ITL agent that interactively learns the meaning of task concepts, the goals, actions, failure conditions, and task-specific terms, for 60 games and puzzles. In our approach, the agent learns hierarchical symbolic representations of task knowledge that enable it to transfer and compose knowledge, analyze and debug multiple interpretations, and communicate with the teacher to resolve ambiguity. Our results show that the agent can correctly generalize, disambiguate, and transfer concepts across variations of language descriptions and world representations, even with distractors present.PHDComputer Science & EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/153434/1/jrkirk_1.pd

Advances in Evolutionary Algorithms

With the recent trends towards massive data sets and significant computational power, combined with evolutionary algorithmic advances evolutionary computation is becoming much more relevant to practice. Aim of the book is to present recent improvements, innovative ideas and concepts in a part of a huge EA field

Context-aware Plan Repair in Environments shared by Multiple Agents

[ES] La monitorización de la ejecución de un plan es crucial para un agente autónomo que realiza su labor en un entorno dinámico, pues influye en su capacidad de reaccionar ante los cambios. Mientras ejecuta su plan puede sufrir un fallo y, en su esfuerzo por solucionarlo, puede interferir sin saberlo con otros agentes que operan en su mismo entorno. Por otra parte, para actuar racionalmente es necesario que el agente sea consciente del contexto y pueda recopilar y ampliar su información a partir de lo que percibe para poder compensar su conocimiento previo parcial o incorrecto del problema y lograr el mejor resultado posible ante las nuevas situaciones que aparecen. El trabajo realizado en esta tesis permite a los agentes autónomos ejecutar sus planes en un entorno dinámico y adaptarse a eventos inesperados y circunstancias desconocidas. Pueden utilizar su percepción del contexto para proporcionar respuestas deliberativas conscientes y ser capaces así de aprovechar las oportunidades que surgen o reparar los fallos sin perturbar a otros agentes. Este trabajo se centra en el desarrollo de una arquitectura independiente del dominio capaz de manejar las necesidades de agentes con este tipo de comportamiento autónomo. Los tres pilares de la arquitectura propuesta los forman el sistema inteligente para la simulación de la ejecución en entornos dinámicos, la adquisición de conocimiento consciente del contexto para ampliar la base de datos del agente y la reparación de planes ante fallos u oportunidades tratando de interferir lo mínimo con los planes de otros agentes. El sistema inteligente de simulación de la ejecución permite al agente representar el plan en una línea de tiempo, actualizar periódicamente su estado interno con información del mundo real y disparar nuevos eventos en momentos concretos. Los eventos se procesan en el contexto del plan; si se detecta un error, el simulador reformula el problema de planificación, invoca de nuevo al planificador y reanuda la ejecución. El simulador es una aplicación de consola y ofrece una interfaz gráfica diseñada específicamente para una aplicación inteligente de turismo. El módulo de adquisición de conocimiento sensible al contexto utiliza operaciones semánticas para aumentar dinámicamente la lista predefinida de tipos de objetos de la tarea de planificación con nuevos tipos relevantes. Esto permite que el agente sea consciente de su entorno, enriquezca el modelo de su tarea y pueda razonar a partir de un conocimiento incompleto. Con todo esto se consigue potenciar la autonomía del sistema y la conciencia del contexto. La novedosa estrategia de reparación de planes le permite a un agente reparar su plan al detectar un fallo de manera responsable con el resto de agentes que comparten su mismo entorno de ejecución. El agente utiliza una nueva métrica, el compromiso del plan, como función heurística para guiar la búsqueda hacia un plan solución comprometido con el plan original, en el sentido de que se trata de respetar los compromisos adquiridos con otros agentes al mismo tiempo que se alcanzan los objetivos originales. En consecuencia, la comunidad de agentes sufrirá menos fallos por cambios bruscos en el entorno o requerirá menos tiempo para ejecutar las acciones correctoras si el fallo es inevitable. Estos tres módulos han sido desarrollados y evaluados en varias aplicaciones como un asistente turístico, una agencia de reparación de electrodomésticos y un asistente del hogar.[CA] El monitoratge de l'execució d'un pla és crucial per a un agent autònom que realitza la seua labor en un entorn dinàmic, perquè influeix en la seua capacitat de reaccionar davant els canvis. Mentre executa el seu pla pot patir una fallada i, en el seu esforç per solucionar-lo, pot interferir sense saber-ho amb altres agents que operen en el seu mateix entorn. D'altra banda, per a actuar racionalment és necessari que l'agent siga conscient del context i puga recopilar i ampliar la seua informació a partir del que percep per a poder compensar el seu coneixement previ parcial o incorrecte del problema i aconseguir el millor resultat possible davant les noves situacions que apareixen. El treball realitzat en aquesta tesi permet als agents autònoms executar els seus plans en un entorn dinàmic i adaptar-se a esdeveniments inesperats i circumstàncies desconegudes. Poden utilitzar la seua percepció del context per a proporcionar respostes deliberatives conscients i ser capaces així d'aprofitar les oportunitats que sorgeixen o reparar les fallades sense pertorbar a altres agents. Aquest treball se centra en el desenvolupament d'una arquitectura independent del domini capaç de manejar les necessitats d'agents amb aquesta mena de comportament autònom. Els tres pilars de l'arquitectura proposada els formen el sistema intel·ligent per a la simulació de l'execució en entorns dinàmics, l'adquisició de coneixement conscient del context per a ampliar la base de dades de l'agent i la reparació de plans davant fallades o oportunitats tractant d'interferir el mínim amb els plans d'altres agents. El sistema intel·ligent de simulació de l'execució permet a l'agent representar el pla en una línia de temps, actualitzar periòdicament el seu estat intern amb informació del món real i disparar nous esdeveniments en moments concrets. Els esdeveniments es processen en el context del pla; si es detecta un error, el simulador reformula el problema de planificació, invoca de nou al planificador i reprén l'execució. El simulador és una aplicació de consola i ofereix una interfície gràfica dissenyada específicament per a una aplicació intel·ligent de turisme. El mòdul d'adquisició de coneixement sensible al context utilitza operacions semàntiques per a augmentar dinàmicament la llista predefinida de tipus d'objectes de la tasca de planificació amb nous tipus rellevants. Això permet que l'agent siga conscient del seu entorn, enriquisca el model de la seua tasca i puga raonar a partir d'un coneixement incomplet. Amb tot això s'aconsegueix potenciar l'autonomia del sistema i la consciència del context. La nova estratègia de reparació de plans li permet a un agent reparar el seu pla en detectar una fallada de manera responsable amb la resta d'agents que comparteixen el seu mateix entorn d'execució. L'agent utilitza una nova mètrica, el compromís del pla, com a funció heurística per a guiar la cerca cap a un pla solució compromés amb el pla original, en el sentit que es tracta de respectar els compromisos adquirits amb altres agents al mateix temps que s'aconsegueixen els objectius originals. En conseqüència, la comunitat d'agents patirà menys fallades per canvis bruscos en l'entorn o requerirà menys temps per a executar les accions correctores si la fallada és inevitable. Aquests tres mòduls han sigut desenvolupats i avaluats en diverses aplicacions com un assistent turístic, una agència de reparació d'electrodomèstics i un assistent de la llar.[EN] Execution Monitoring is crucial for the success of an autonomous agent executing a plan in a dynamic environment as it influences its ability to react to changes. While executing its plan in a dynamic world, it may suffer a failure and, in its endeavour to fix the problem, it may unknowingly disrupt other agents operating in the same environment. Additionally, being rational requires the agent to be context-aware, gather information and extend what is known from what is perceived to compensate for partial or incorrect prior knowledge and achieve the best possible outcome in various novel situations. The work carried out in this PhD thesis allows the autonomous agents executing a plan in a dynamic environment to adapt to unexpected events and unfamiliar circumstances, utilise their perception of context and provide context-aware deliberative responses for seizing an opportunity or repairing a failure without disrupting other agents. This work is focused on developing a domain-independent architecture capable of handling the requirements of such autonomous behaviour. The architecture pillars are the intelligent system for execution simulation in a dynamic environment, the context-aware knowledge acquisition for planning applications and the plan commitment repair. The intelligent system for execution simulation in a dynamic environment allows the agent to transform the plan into a timeline, periodically update its internal state with real-world information and create timed events. Events are processed in the context of the plan; if a failure occurs, the simulator reformulates the planning problem, reinvokes a planner and resumes the execution. The simulator is a console application and has a GUI designed specifically for smart tourism. The context-aware knowledge acquisition module utilises semantic operations to dynamically augment the predefined list of object types of the planning task with relevant new object types. This allows the agent to be context-aware of the environment and the task and reason with incomplete knowledge, boosting the system's autonomy and context-awareness. The novel plan commitment repair strategy among multiple agents sharing the same execution environment allows the agent to repair its plan responsibly when a failure is detected. The agent utilises a new metric, plan commitment, as a heuristic to guide the search for the most committed repair plan to the original plan from the perspective of commitments made to other agents whilst achieving the original goals. Consequently, the community of agents will suffer fewer failures due to the sudden changes or will have less lost time if the failure is inevitable. All these developed modules were investigated and evaluated in several applications, such as a tourist assistant, a kitchen appliance repair agency and a living home assistant.Babli, M. (2023). Context-aware Plan Repair in Environments shared by Multiple Agents [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/19868

Empirical modelling as a new paradigm for educational technology

Educational technology has yet to deliver the benefits or successes that were expected in educational practice, especially in relation to issues other than the communication and delivery of teaching materials. Evidence suggests that these difficulties stem from the mismatch between formalised virtual learning environments and everyday sensemaking and between the rich potential for enhanced learning afforded by new technology and the constraints of old-style educational practice. In addressing this mismatch, some commentators suggest that the primary need is for a new culture of educational practice-and even that such a culture is already emerging, and others identify the need for a new paradigm for educational technology. The aim of this thesis is to explore the potential for a new paradigm for educational technology based on the principles and tools of Empirical Modelling (see http://dcs.warwick.ac.uk/modelling). The thesis builds upon previous research on Empirical Modelling as a constructionist approach to learning, and in particular Roe's doctoral thesis 'Computers for learning: an Empirical Modelling perspective'. Roe's treatment of Empirical Modelling can be viewed as generalising the use of spreadsheets for learning through applying 'programming by dependency' within the framework of existing educational practice. In contrast, this thesis is concerned at a more fundamental level with the contribution that Empirical Modelling can make to technology enhanced learning that may lead to new educational practices. In particular, it identifies eight significant characteristics of learning that are well-matched to Empirical Modelling activity, and associates these with experimental, flexible and meaningful strands in learning. The credentials of Empirical Modelling as a potential new foundation for educational technology are enhanced by demonstrating that Empirical Modelling is radically different from traditional software development and use. It provides a methodology for modelling with dependency that is more closely related to the use of spreadsheets for learning. The thesis elaborates on the relationship between Empirical Modelling and learning in a variety of different contexts, ways and applications. Three examples drawn from computer science higher education are explored to emphasise the experimental, flexible and meaningful characteristics of Empirical Modelling. This discussion of Empirical Modelling in a specific educational context is complemented by an investigation of its relevance to learning in a wider context, with reference to a broad range of subjects, to specific issues in language learning, and to the topics of lifelong learning and collaborative learning. Although the application of Empirical Modelling for learning is as yet too immature for large scale empirical studies, its potential is evaluated using informal empirical evidence arising from Empirical Modelling practice at Warwick. The sources for this evaluation are well-established teaching activities relating to Empirical Modelling in Computer Science at the University of 'Warwick, comprising an introductory module and a number of final year undergraduate projects. The thesis concludes by considering the extent to which Empirical Modelling can go beyond the support for constructionism envisaged by Roe, to address the broader agenda of supporting constructivist learning using computers. To this end, a close relationship between Empirical Modelling and a vision of constructivism recently set out by Bruno Latour in his paper 'The Promises of Constructivism' is demonstrated

ONTOLOGY MAPPING: TOWARDS SEMANTIC INTEROPERABILITY IN DISTRIBUTED AND HETEROGENEOUS ENVIRONMENTS

The World Wide Web (WWW) now is widely used as a universal medium for information exchange. Semantic interoperability among different information systems in the WWW is limited due to information heterogeneity, and the non semantic nature of HTML and URLs. Ontologies have been suggested as a way to solve the problem of information heterogeneity by providing formal, explicit definitions of data and reasoning ability over related concepts. Given that no universal ontology exists for the WWW, work has focused on finding semantic correspondences between similar elements of different ontologies, i.e., ontology mapping. Ontology mapping can be done either by hand or using automated tools. Manual mapping becomes impractical as the size and complexity of ontologies increases. Full or semi-automated mapping approaches have been examined by several research studies. Previous full or semi-automated mapping approaches include analyzing linguistic information of elements in ontologies, treating ontologies as structural graphs, applying heuristic rules and machine learning techniques, and using probabilistic and reasoning methods etc. In this paper, two generic ontology mapping approaches are proposed. One is the PRIOR+ approach, which utilizes both information retrieval and artificial intelligence techniques in the context of ontology mapping. The other is the non-instance learning based approach, which experimentally explores machine learning algorithms to solve ontology mapping problem without requesting any instance. The results of the PRIOR+ on different tests at OAEI ontology matching campaign 2007 are encouraging. The non-instance learning based approach has shown potential for solving ontology mapping problem on OAEI benchmark tests

Empirical modelling as a new paradigm for educational technology

Educational technology has yet to deliver the benefits or successes that were expected in educational practice, especially in relation to issues other than the communication and delivery of teaching materials. Evidence suggests that these difficulties stem from the mismatch between formalised virtual learning environments and everyday sensemaking and between the rich potential for enhanced learning afforded by new technology and the constraints of old-style educational practice. In addressing this mismatch, some commentators suggest that the primary need is for a new culture of educational practice-and even that such a culture is already emerging, and others identify the need for a new paradigm for educational technology. The aim of this thesis is to explore the potential for a new paradigm for educational technology based on the principles and tools of Empirical Modelling (see http://dcs.warwick.ac.uk/modelling). The thesis builds upon previous research on Empirical Modelling as a constructionist approach to learning, and in particular Roe's doctoral thesis 'Computers for learning: an Empirical Modelling perspective'. Roe's treatment of Empirical Modelling can be viewed as generalising the use of spreadsheets for learning through applying 'programming by dependency' within the framework of existing educational practice. In contrast, this thesis is concerned at a more fundamental level with the contribution that Empirical Modelling can make to technology enhanced learning that may lead to new educational practices. In particular, it identifies eight significant characteristics of learning that are well-matched to Empirical Modelling activity, and associates these with experimental, flexible and meaningful strands in learning. The credentials of Empirical Modelling as a potential new foundation for educational technology are enhanced by demonstrating that Empirical Modelling is radically different from traditional software development and use. It provides a methodology for modelling with dependency that is more closely related to the use of spreadsheets for learning. The thesis elaborates on the relationship between Empirical Modelling and learning in a variety of different contexts, ways and applications. Three examples drawn from computer science higher education are explored to emphasise the experimental, flexible and meaningful characteristics of Empirical Modelling. This discussion of Empirical Modelling in a specific educational context is complemented by an investigation of its relevance to learning in a wider context, with reference to a broad range of subjects, to specific issues in language learning, and to the topics of lifelong learning and collaborative learning. Although the application of Empirical Modelling for learning is as yet too immature for large scale empirical studies, its potential is evaluated using informal empirical evidence arising from Empirical Modelling practice at Warwick. The sources for this evaluation are well-established teaching activities relating to Empirical Modelling in Computer Science at the University of 'Warwick, comprising an introductory module and a number of final year undergraduate projects. The thesis concludes by considering the extent to which Empirical Modelling can go beyond the support for constructionism envisaged by Roe, to address the broader agenda of supporting constructivist learning using computers. To this end, a close relationship between Empirical Modelling and a vision of constructivism recently set out by Bruno Latour in his paper 'The Promises of Constructivism' is demonstrated.EThOS - Electronic Theses Online ServiceGBUnited Kingdo