Building a poker playing agent based on game logs using supervised learning

Tese de mestrado integrado. Engenharia Informática e Computação. Faculdade de Engenharia. Universidade do Porto. 201

Reinforcement learning-based AI assistant and VR play therapy game for children with Down syndrome bound to wheelchairs

Some of the most significant computational ideas in neuroscience for learning behavior in response to reward and penalty are reinforcement learning algorithms. This technique can be used to train an artificial intelligent (AI) agent to serve as a virtual assistant and a helper. The goal of this study is to determine whether combining a reinforcement learning-based Virtual AI assistant with play therapy. It can benefit wheelchair-bound youngsters with Down syndrome. This study aims to employ play therapy methods and Reinforcement Learning (RL) agents to aid children with Down syndrome and help them enhance their abilities like physical and mental skills by playing games with them. This Agent is designed to be smart enough to analyze each patient's lack of ability and provide a specific set of challenges in the game to improve that ability. Increasing the game's difficulty can help players develop these skills. The agent should be able to assess each player's skill gap and tailor the game to them accordingly. The agent's job is not to make the patient victorious but to boost their morale and skill sets in areas like physical activities, intelligence, and social interaction. The primary objective is to improve the player's physical activities such as muscle reflexes, motor controls and hand-eye coordination. Here, the study concentrates on the employment of several distinct techniques for training various models. This research focuses on comparing the reinforcement learning algorithms like the Deep Q-Learning Network, QR-DQN, A3C and PPO-Actor Critic. This study demonstrates that when compared to other reinforcement algorithms, the performance of the AI helper agent is at its highest when it is trained with PPO-Actor Critic and A3C. The goal is to see if children with Down syndrome who are wheelchair-bound can benefit by combining reinforcement learning with play therapy to increase their mobility

Player Behavior Modeling In Video Games

Player Behavior Modeling in Video Games In this research, we study players’ interactions in video games to understand player behavior. The first part of the research concerns predicting the winner of a game, which we apply to StarCraft and Destiny. We manage to build models for these games which have reasonable to high accuracy. We also investigate which features of a game comprise strong predictors, which are economic features and micro commands for StarCraft, and key shooter performance metrics for Destiny, though features differ between different match types. The second part of the research concerns distinguishing playing styles of players of StarCraft and Destiny. We find that we can indeed recognize different styles of playing in these games, related to different match types. We relate these different playing styles to chance of winning, but find that there are no significant differences between the effects of different playing styles on winning. However, they do have an effect on the length of matches. In Destiny, we also investigate what player types are distinguished when we use Archetype Analysis on playing style features related to change in performance, and find that the archetypes correspond to different ways of learning. In the final part of the research, we investigate to what extent playing styles are related to different demographics, in particular to national cultures. We investigate this for four popular Massively multiplayer online games, namely Battlefield 4, Counter-Strike, Dota 2, and Destiny. We found that playing styles have relationship with nationality and cultural dimensions, and that there are clear similarities between the playing styles of similar cultures. In particular, the Hofstede dimension Individualism explained most of the variance in playing styles between national cultures for the games that we examined

A game-based approach towards human augmented image annotation.

PhDImage annotation is a difficult task to achieve in an automated way. In this thesis, a human-augmented approach to tackle this problem is discussed and suitable strategies are derived to solve it. The proposed technique is inspired by human-based computation in what is called “human-augmented” processing to overcome limitations of fully automated technology for closing the semantic gap. The approach aims to exploit what millions of individual gamers are keen to do, i.e. enjoy computer games, while annotating media. In this thesis, the image annotation problem is tackled by a game based framework. This approach combines image processing and a game theoretic model to gather media annotations. Although the proposed model behaves similar to a single player game model, the underlying approach has been designed based on a two-player model which exploits the player’s contribution to the game and previously recorded players to improve annotations accuracy. In addition, the proposed framework is designed to predict the player’s intention through Markovian and Sequential Sampling inferences in order to detect cheating and improve annotation performances. Finally, the proposed techniques are comprehensively evaluated with three different image datasets and selected representative results are reported

Remedies for Robots

What happens when artificially intelligent robots misbehave? The question is not just hypothetical. As robotics and artificial intelligence systems increasingly integrate into our society, they will do bad things. We seek to explore what remedies the law can and should provide once a robot has caused harm. Remedies are sometimes designed to make plaintiffs whole by restoring them to the condition they would have been in “but for” the wrong. But they can also contain elements of moral judgment, punishment, and deterrence. In other instances, the law may order defendants to do (or stop doing) something unlawful or harmful. Each of these goals of remedies law, however, runs into difficulties when the bad actor in question is neither a person nor a corporation but a robot. We might order a robot—or, more realistically, the designer or owner of the robot—to pay for the damages it causes. But it turns out to be much harder for a judge to “order” a robot, rather than a human, to engage in or refrain from certain conduct. Robots can’t directly obey court orders not written in computer code. And bridging the translation gap between natural language and code is often harder than we might expect. This is particularly true of modern artificial intelligence techniques that empower machines to learn and modify their decision-making over time. If we don’t know how the robot “thinks,” we won’t know how to tell it to behave in a way likely to cause it to do what we actually want it to do. Moreover, if the ultimate goal of a legal remedy is to encourage good behavior or discourage bad behavior, punishing owners or designers for the behavior of their robots may not always make sense—if only for the simple reason that their owners didn’t act wrongfully in any meaningful way. The same problem affects injunctive relief. Courts are used to ordering people and companies to do (or stop doing) certain things, with a penalty of contempt of court for noncompliance. But ordering a robot to abstain from certain behavior won’t be trivial in many cases. And ordering it to take affirmative acts may prove even more problematic. In this Article, we begin to think about how we might design a system of remedies for robots. Robots will require us to rethink many of our current doctrines. They also offer important insights into the law of remedies we already apply to people and corporations

Everything You Never Wanted to Know about Trolls:An Interdisplinary Exploration of the Who's, What's, and Why's of Trolling in Online Games

Summary Within the world of online gaming, trolling has become a regular menace. While gamers try to connect and socialize with one another, or even simply play the game, there are other gamers – trolls – on the prowl for an entirely different kind of good time, one in which they are enjoying themselves at the expense of everyone else (Chapters 2 and 3). Although trolling is common, and mass-media has latched onto it as a hot topic, it is only recently that the academic community has begun to take a serious look at how trolling occurs in and affects the gaming community at large. However, a lot of this literature is either descriptive in nature (see Thacker & Griffiths, 2012), or jumps ahead to prevention (see Cheng et al., 2017) without taking a deeper look at more than a single underlying motivation at a time. In short, there is a complex and prolific phenomenon happening online, but the research on it is only emerging. This dissertation’s goal is to take a deeper look at trolling as a phenomenon, beyond what has been done so far. More specifically, I aim to figure out a) what trolling is, b) why people do it, and c) who helps and who hinders trolling in online games. To do this, I took four different perspectives: the troll’s (Chapter 2), the researcher’s (Chapter 3), the victim’s (Chapter 4), and the bystander’s (Chapter 5). The purpose of Chapter 2 is to give the troll’s perspective on trolling, something that researchers had yet to do at the time. To do this, I interviewed 22 people who said that they had a history of trolling in online games. More specifically, I asked them about times they witnessed, were victims of, or perpetrated trolling, as well as what they thought about how the gaming community dealt with and felt about trolls and trolling. My goal with these interviews was threefold: I wanted to figure out a) what trolls consider trolling, b) what motivates them to do it, and c) the role of everyone else in game when it comes to encouraging or discouraging more trolling. What I found was that although trolling was almost universally considered a negative part of online gaming culture, and all the trolls in our group of participants started as victims of trolls before becoming trolls themselves, the online community neither encourages nor discourages it, making it an asocial activity. The next chapter allowed me to look at an archive of trolling incidents to find patterns in the way that different people involved in real-life trolling incidents communicate with one another. This public online archive consisted of 10,000 reported incidents of trolling in the popular online game League of Legends, and it included game data like player statistics, as well as everything all the players involved said during the game. Once the data was properly cleaned and prepared, myself and my co-author, Dr. Rianne Conijn, analysed the chat logs in two different ways: structural topic modelling (STM), and a traditional dictionary-based content analysis. In this way, we were able to see what characterized all the different actors – the troll, their victim(s), and the bystanders – and what was similar when it came to their messages. All this information was then compared to what existed already in literature used to describe trolls and trolling and complement what I had learned about trolls from Chapter 2. The key finding was that trolls and their teammates actually share a lot of the negative speech patterns (e.g., profanity, negative emotional content) normally associated with only trolls. Practically, this means that we have to be extremely careful as researchers when labelling trolls for the purpose of study, as we could very easily be falsely labelling victims. After speaking to trolls and looking at trolling interactions broadly, Chapter 4 focuses intently on the victim and their personal experience in a trolling simulation, taking into account their cultural background and values. It is also the first study to directly compare and contrast two different types of trolling: verbal (flaming) and behavioural (ostracism). They are both really common online occurrences, so the participants could easily relate, but they are extremely different in how they are executed, with flaming being vicious insults and ostracism being totally ignoring a person. Our participants were either Dutch, Pakistani, or Taiwanese, so that we could also look at how people from vastly different cultural backgrounds would react to – behaviourally and emotionally – the different kinds of trolling in the study. We simulated a trolling experience by putting our participants in a virtual game of catch with two computerized co-players, who they were led to believe were real people of either the same nationality or a minority member (e.g., a Moroccan immigrant in the Netherlands), who I had programmed to either troll them or silently watch the trolling happen. We found that there are indeed cultural differences when it comes to reactions, as well as differences between reactions to the two trolling types, but the core take-away is that future trolling interventions have to take into account the cultures of the target population as well as the specific type of trolling they are trying to fix or prevent in order to be effective. In the penultimate chapter, I shift the focus one last time to bystanders by putting participants in a game of League of Legends with two confederates who would troll one another throughout the game. This study’s goal was to see what motivated gamers to report trolls to an authority figure (the game developer) using the game’s built-in reporting functions, as the results of Chapter 2’s study suggested that this was an effective trolling deterrent. It is also, according to the results of the same study, the least-used recourse by bystanders faced with trolls in the proverbial wild. We found that how warm and friendly the troll was perceived to be and how competent the victim was perceived to be were what determined whether the participant reported our fake troll or not. A more competent victim and a less warm troll lead to more reports. To conclude, there is still a lot more to learn about trolls and trolling, but the field is farther along now than when this project started in 2015. There is a broad definition developed that encompasses most of the descriptive literature on trolling in games thus far. We also now know that there is the indication of a trolling cycle that requires further exploration. This is particularly important to know when it comes to the world of game development, as knowing the cycle exists allows for multiple points of intervention in order to protect their customers. Finally, this dissertation has shown the complexity of not just trolls – who are often portrayed in the media as one-dimensional antagonists – but also of everyone else involved in trolling interactions. Trolls, victims, and bystanders are all multi-faceted humans, and trolling, like all interactions, is an intricate social dance that deserves to be studied in even further depth in the future than what I have done here

The emerging landscape of Social Media Data Collection: anticipating trends and addressing future challenges

[spa] Las redes sociales se han convertido en una herramienta poderosa para crear y compartir contenido generado por usuarios en todo internet. El amplio uso de las redes sociales ha llevado a generar una enorme cantidad de información, presentando una gran oportunidad para el marketing digital. A través de las redes sociales, las empresas pueden llegar a millones de consumidores potenciales y capturar valiosos datos de los consumidores, que se pueden utilizar para optimizar estrategias y acciones de marketing. Los beneficios y desafíos potenciales de utilizar las redes sociales para el marketing digital también están creciendo en interés entre la comunidad académica. Si bien las redes sociales ofrecen a las empresas la oportunidad de llegar a una gran audiencia y recopilar valiosos datos de los consumidores, el volumen de información generada puede llevar a un marketing sin enfoque y consecuencias negativas como la sobrecarga social. Para aprovechar al máximo el marketing en redes sociales, las empresas necesitan recopilar datos confiables para propósitos específicos como vender productos, aumentar la conciencia de marca o fomentar el compromiso y para predecir los comportamientos futuros de los consumidores. La disponibilidad de datos de calidad puede ayudar a construir la lealtad a la marca, pero la disposición de los consumidores a compartir información depende de su nivel de confianza en la empresa o marca que lo solicita. Por lo tanto, esta tesis tiene como objetivo contribuir a la brecha de investigación a través del análisis bibliométrico del campo, el análisis mixto de perfiles y motivaciones de los usuarios que proporcionan sus datos en redes sociales y una comparación de algoritmos supervisados y no supervisados para agrupar a los consumidores. Esta investigación ha utilizado una base de datos de más de 5,5 millones de colecciones de datos durante un período de 10 años. Los avances tecnológicos ahora permiten el análisis sofisticado y las predicciones confiables basadas en los datos capturados, lo que es especialmente útil para el marketing digital. Varios estudios han explorado el marketing digital a través de las redes sociales, algunos centrándose en un campo específico, mientras que otros adoptan un enfoque multidisciplinario. Sin embargo, debido a la naturaleza rápidamente evolutiva de la disciplina, se requiere un enfoque bibliométrico para capturar y sintetizar la información más actualizada y agregar más valor a los estudios en el campo. Por lo tanto, las contribuciones de esta tesis son las siguientes. En primer lugar, proporciona una revisión exhaustiva de la literatura sobre los métodos para recopilar datos personales de los consumidores de las redes sociales para el marketing digital y establece las tendencias más relevantes a través del análisis de artículos significativos, palabras clave, autores, instituciones y países. En segundo lugar, esta tesis identifica los perfiles de usuario que más mienten y por qué. Específicamente, esta investigación demuestra que algunos perfiles de usuario están más inclinados a cometer errores, mientras que otros proporcionan información falsa intencionalmente. El estudio también muestra que las principales motivaciones detrás de proporcionar información falsa incluyen la diversión y la falta de confianza en las medidas de privacidad y seguridad de los datos. Finalmente, esta tesis tiene como objetivo llenar el vacío en la literatura sobre qué algoritmo, supervisado o no supervisado, puede agrupar mejor a los consumidores que proporcionan sus datos en las redes sociales para predecir su comportamiento futuro

To Affinity and Beyond: Interactive Digital Humans as a Human Computer Interface

The field of human computer interaction is increasingly exploring the use of more natural, human-like user interfaces to build intelligent agents to aid in everyday life. This is coupled with a move to people using ever more realistic avatars to represent themselves in their digital lives. As the ability to produce emotionally engaging digital human representations is only just now becoming technically possible, there is little research into how to approach such tasks. This is due to both technical complexity and operational implementation cost. This is now changing as we are at a nexus point with new approaches, faster graphics processing and enabling new technologies in machine learning and computer vision becoming available. I articulate the issues required for such digital humans to be considered successfully located on the other side of the phenomenon known as the Uncanny Valley. My results show that a complex mix of perceived and contextual aspects affect the sense making on digital humans and highlights previously undocumented effects of interactivity on the affinity. Users are willing to accept digital humans as a new form of user interface and they react to them emotionally in previously unanticipated ways. My research shows that it is possible to build an effective interactive digital human that crosses the Uncanny Valley. I directly explore what is required to build a visually realistic digital human as a primary research question and I explore if such a realistic face provides sufficient benefit to justify the challenges involved in building it. I conducted a Delphi study to inform the research approaches and then produced a complex digital human character based on these insights. This interactive and realistic digital human avatar represents a major technical undertaking involving multiple teams around the world. Finally, I explored a framework for examining the ethical implications and signpost future research areas

Analysis of human-computer interaction time series using Deep Learning

Dissertação de mestrado integrado em Engenharia InformáticaThe collection and use of data resulting from human-computer interaction are becoming more and more common. These have been allowing for the birth of intelligent systems that extract powerful knowledge, potentially improving the user experience or even originating various digital services. With the rapid scientific advancements that have been taking place in the field of Deep Learning, it is convenient to review the underlying techniques currently used in these systems. In this work, we propose an approach to the general task of analyzing such interactions in the form of time series, using Deep Learning. We then rely on this approach to develop an anti-cheating system for video games using only keyboard and mouse input data. This system can work with any video game, and with minor adjustments, it can be easily adapted to new platforms (such as mobile and gaming consoles). Experiments suggest that analyzing HCI time series data with deep learning yields better results while providing solutions that do not rely highly on domain knowledge as traditional systems.A recolha e a utilização de dados resultantes da interação humano-computador estão a tornar-se cada vez mais comuns. Estas têm permitido o surgimento de sistemas inteligentes capazes de extrair conhecimento ex tremamente útil, potencialmente melhorando a experiência do utilizador ou mesmo originando diversos serviços digitais. Com os acelerados avanços científicos na área do Deep Learning, torna-se conveniente rever as técni cas subjacentes a estes sistemas. Neste trabalho, propomos uma abordagem ao problema geral de analisar tais interações na forma de séries temporais, utilizando Deep Learning. Apoiamo-nos então nesta abordagem para desenvolver um sistema de anti-cheating para videojogos, utilizando apenas dados de input de rato e teclado. Este sistema funciona com qualquer jogo e pode, com pequenos ajustes, ser adaptado para novas plataformas (como dispositivos móveis ou consolas). As experiências sugerem que analisar dados de séries temporais de interação humano-computador pro duz melhores resultados, disponibilizando soluções que não são altamente dependentes de conhecimento de domínio como sistemas tradicionais