Pedagogical Possibilities for the N-Puzzle Problem

In this paper we present work on a project funded by the National Science Foundation with a goal of unifying the Artificial Intelligence (AI) course around the theme of machine learning. Our work involves the development and testing of an adaptable framework for the presentation of core AI topics that emphasizes the relationship between AI and computer science. Several hands-on laboratory projects that can be closely integrated into an introductory AI course have been developed. We present an overview of one of the projects and describe the associated curricular materials that have been developed. The project uses machine learning as a theme to unify core AI topics in the context of the N-puzzle game. Games provide a rich framework to introduce students to search fundamentals and other core AI concepts. The paper presents several pedagogical possibilities for the N-puzzle game, the rich challenge it offers, and summarizes our experiences using it

Problem Solving for Industry

This project seeks to use reinforcement learning to develop AI agents used to controlled NPCs in video game worlds that are capable of mastering decision tasks in their video game environments. Our job will be to develop algorithms and methods that can effectively train the AI agents using Reinforcement learning, which can be used in various gaming environments and scenarios such as racing games and first-person shooters. We then market these agents to video game developers for use in their game worlds. The developer can use our agents as-is in their game without modifications or they can train them further, using our algorithms, to tune the AI agents with various behaviours and capability with minimal or no need to write the code themselves. With the use of reinforcement learning, our AI agents will learn using trial and error with rewards used to provide feedback to the AI. Over time the AI will master its environment and other AI and even possibly interaction with the human-gamer. This will produce AI controlled NPCs that behave and interact convincingly with their environments and the player, promoting player immersions while reducing developer workload

AI Education: Birds of a Feather

Games are beautifully crafted microworlds that invite players to explore complex terrains that spring into existence from even simple rules. As AI educators, games can offer fun ways of teaching important concepts and techniques. Just as Martin Gardner employed games and puzzles to engage both amateurs and professionals in the pursuit of Mathematics, a well-chosen game or puzzle can provide a catalyst for AI learning and research. [excerpt

An investigation of two real time machine learning techniques that could enhance the adaptability of game AI agents

Developers strive to create innovative Artificial Intelligence (AI) behaviour in their games as a key selling point. Machine Learning is an area of AI that looks at how applications and agents can be programmed to learn their own behaviour without the need to manually design and implement each aspect of it. Machine learning methods have been utilised infrequently within games and are usually trained to learn offline before the game is released to the players. In order to investigate new ways AI could be applied innovatively to games it is wise to explore how machine learning methods could be utilised in real-time as the game is played, so as to allow AI agents to learn directly from the player or their environment. Two machine learning methods were implemented into a simple 2D Fighter test game to allow the agents to fully showcase their learned behaviour as the game is played. The methods chosen were: Q-Learning and an NGram based system. It was found that N-Grams and QLearning could significantly benefit game developers as they facilitate fast, realistic learning at run-time

Improving Computer Game Bots\u27 behavior using Q-Learning

In modern computer video games, the quality of artificial characters plays a prominent role in the success of the game in the market. The aim of intelligent techniques, termed game AI, used in these games is to provide an interesting and challenging game play to a game player. Being highly sophisticated, these games present game developers with similar kind of requirements and challenges as faced by academic AI community. The game companies claim to use sophisticated game AI to model artificial characters such as computer game bots, intelligent realistic AI agents. However, these bots work via simple routines pre-programmed to suit the game map, game rules, game type, and other parameters unique to each game. Mostly, illusive intelligent behaviors are programmed using simple conditional statements and are hard-coded in the bots\u27 logic. Moreover, a game programmer has to spend considerable time configuring crisp inputs for these conditional statements. Therefore, we realize a need for machine learning techniques to dynamically improve bots\u27 behavior and save precious computer programmers\u27 man-hours. So, we selected Q-learning, a reinforcement learning technique, to evolve dynamic intelligent bots, as it is a simple, efficient, and online learning algorithm. Machine learning techniques such as reinforcement learning are know to be intractable if they use a detailed model of the world, and also requires tuning of various parameters to give satisfactory performance. Therefore, for this research we opt to examine Q-learning for evolving a few basic behaviors viz. learning to fight, and planting the bomb for computer game bots. Furthermore, we experimented on how bots would use knowledge learned from abstract models to evolve its behavior in more detailed model of the world. Bots evolved using these techniques would become more pragmatic, believable and capable of showing human-like behavior. This will provide more realistic feel to the game and provide game programmers with an efficient learning technique for programming these bots

Natively Implementing Deep Reinforcement Learning into a Game Engine

Artificial intelligence (AI) increases the immersion that players can have while playing games. Modern game engines, a middleware software used to create games, implement simple AI behaviors that developers can use. Advanced AI behaviors must be implemented manually by game developers, which decreases the likelihood of game developers using advanced AI due to development overhead. A custom game engine and custom AI architecture that handled deep reinforcement learning was designed and implemented. Snake was created using the custom game engine to test the feasibility of natively implementing an AI architecture into a game engine. A snake agent was successfully trained using the AI architecture, but the learned behavior was suboptimal. Although the learned behavior was suboptimal, the AI architecture was successfully implemented into a custom game engine because a behavior was successfully learned

Monte Carlo Approaches to Parameterized Poker Squares

The paper summarized a variety of Monte Carlo approaches employed in the top three performing entries to the Parameterized Poker Squares NSG Challenge competition. In all cases AI players benefited from real-time machine learning and various Monte Carlo game-tree search techniques