372 research outputs found
Pitako -- Recommending Game Design Elements in Cicero
Recommender Systems are widely and successfully applied in e-commerce. Could
they be used for design? In this paper, we introduce Pitako1, a tool that
applies the Recommender System concept to assist humans in creative tasks. More
specifically, Pitako provides suggestions by taking games designed by humans as
inputs, and recommends mechanics and dynamics as outputs. Pitako is implemented
as a new system within the mixed-initiative AI-based Game Design Assistant,
Cicero. This paper discusses the motivation behind the implementation of Pitako
as well as its technical details and presents usage examples. We believe that
Pitako can influence the use of recommender systems to help humans in their
daily tasks.Comment: Paper accepted in the IEEE Conference on Games 2019 (COG 2019
Evaluation of a Recommender System for Assisting Novice Game Designers
Game development is a complex task involving multiple disciplines and
technologies. Developers and researchers alike have suggested that AI-driven
game design assistants may improve developer workflow. We present a recommender
system for assisting humans in game design as well as a rigorous human subjects
study to validate it. The AI-driven game design assistance system suggests game
mechanics to designers based on characteristics of the game being developed. We
believe this method can bring creative insights and increase users'
productivity. We conducted quantitative studies that showed the recommender
system increases users' levels of accuracy and computational affect, and
decreases their levels of workload.Comment: The 15th AAAI Conference on Artificial Intelligence and Interactive
Digital Entertainment (AIIDE 19
General Video Game AI: A Multitrack Framework for Evaluating Agents, Games, and Content Generation Algorithms
General Video Game Playing (GVGP) aims at designing an agent that is capable
of playing multiple video games with no human intervention. In 2014, The
General Video Game AI (GVGAI) competition framework was created and released
with the purpose of providing researchers a common open-source and easy to use
platform for testing their AI methods with potentially infinity of games
created using Video Game Description Language (VGDL). The framework has been
expanded into several tracks during the last few years to meet the demand of
different research directions. The agents are required either to play multiple
unknown games with or without access to game simulations, or to design new game
levels or rules. This survey paper presents the VGDL, the GVGAI framework,
existing tracks, and reviews the wide use of GVGAI framework in research,
education and competitions five years after its birth. A future plan of
framework improvements is also described.Comment: 20 pages, 1 figure, accepted by IEEE To
Enhancing the Monte Carlo Tree Search Algorithm for Video Game Testing
In this paper, we study the effects of several Monte Carlo Tree Search (MCTS)
modifications for video game testing. Although MCTS modifications are highly
studied in game playing, their impacts on finding bugs are blank. We focused on
bug finding in our previous study where we introduced synthetic and human-like
test goals and we used these test goals in Sarsa and MCTS agents to find bugs.
In this study, we extend the MCTS agent with several modifications for game
testing purposes. Furthermore, we present a novel tree reuse strategy. We
experiment with these modifications by testing them on three testbed games,
four levels each, that contain 45 bugs in total. We use the General Video Game
Artificial Intelligence (GVG-AI) framework to create the testbed games and
collect 427 human tester trajectories using the GVG-AI framework. We analyze
the proposed modifications in three parts: we evaluate their effects on bug
finding performances of agents, we measure their success under two different
computational budgets, and we assess their effects on human-likeness of the
human-like agent. Our results show that MCTS modifications improve the bug
finding performance of the agents
Machine Learning based Procedural Content Generation in Semantic Choreography
BeatMania is a rhythm-action game where players press buttons in response to keysound events to generate music. Rhythm-action game charts (the sequence of keysound events) have traditionally been human authored, since each song level must be creatively organized and correspond an overall pattern or theme. A deep neural network approach is proposed for rhythm-action game chart creation, and a method of level evaluation for co-creative AI is defined. That is, given an arbitrary piece of music, human users can generate BeatMania charts as well as give input to an AI collaborator. The problem is divided into two parts: autonomous chart generation and design interaction. For the chart generation process, a combination of features that include grouping information and audio sample labels are incorporated into an artificial neural network. For the design interaction, principal component analysis is utilized for a proposed reinforcement learning model. The co-creative tool is tested against Markov Chain and LSTM baselines via human trials.Undergraduat
Automated Video Game Testing Using Synthetic and Human-Like Agents
In this paper, we present a new methodology that employs tester agents to
automate video game testing. We introduce two types of agents -synthetic and
human-like- and two distinct approaches to create them. Our agents are derived
from Reinforcement Learning (RL) and Monte Carlo Tree Search (MCTS) agents, but
focus on finding defects. The synthetic agent uses test goals generated from
game scenarios, and these goals are further modified to examine the effects of
unintended game transitions. The human-like agent uses test goals extracted by
our proposed multiple greedy-policy inverse reinforcement learning (MGP-IRL)
algorithm from tester trajectories. MGPIRL captures multiple policies executed
by human testers. These testers' aims are finding defects while interacting
with the game to break it, which is considerably different from game playing.
We present interaction states to model such interactions. We use our agents to
produce test sequences, run the game with these sequences, and check the game
for each run with an automated test oracle. We analyze the proposed method in
two parts: we compare the success of human-like and synthetic agents in bug
finding, and we evaluate the similarity between humanlike agents and human
testers. We collected 427 trajectories from human testers using the General
Video Game Artificial Intelligence (GVG-AI) framework and created three games
with 12 levels that contain 45 bugs. Our experiments reveal that human-like and
synthetic agents compete with human testers' bug finding performances.
Moreover, we show that MGP-IRL increases the human-likeness of agents while
improving the bug finding performance
How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review
Context: Machine Learning (ML) has been at the heart of many innovations over
the past years. However, including it in so-called 'safety-critical' systems
such as automotive or aeronautic has proven to be very challenging, since the
shift in paradigm that ML brings completely changes traditional certification
approaches.
Objective: This paper aims to elucidate challenges related to the
certification of ML-based safety-critical systems, as well as the solutions
that are proposed in the literature to tackle them, answering the question 'How
to Certify Machine Learning Based Safety-critical Systems?'.
Method: We conduct a Systematic Literature Review (SLR) of research papers
published between 2015 to 2020, covering topics related to the certification of
ML systems. In total, we identified 217 papers covering topics considered to be
the main pillars of ML certification: Robustness, Uncertainty, Explainability,
Verification, Safe Reinforcement Learning, and Direct Certification. We
analyzed the main trends and problems of each sub-field and provided summaries
of the papers extracted.
Results: The SLR results highlighted the enthusiasm of the community for this
subject, as well as the lack of diversity in terms of datasets and type of
models. It also emphasized the need to further develop connections between
academia and industries to deepen the domain study. Finally, it also
illustrated the necessity to build connections between the above mention main
pillars that are for now mainly studied separately.
Conclusion: We highlighted current efforts deployed to enable the
certification of ML based software systems, and discuss some future research
directions.Comment: 60 pages (92 pages with references and complements), submitted to a
journal (Automated Software Engineering). Changes: Emphasizing difference
traditional software engineering / ML approach. Adding Related Works, Threats
to Validity and Complementary Materials. Adding a table listing papers
reference for each section/subsection
- …