11 research outputs found

    A Profitable Online Poker Agent

    Get PDF
    Jogos de informação incompleta tais como poker são uma fonte contínua de estudo e pesquisa no âmbito da inteligência artificial. No poker problemas como: modelação de oponentes; gestão de riscos e detecção de bluffs representam um desafio. O desenvolvimento de agentes capazes de considerar esses problemas e realizar cálculos probabilísticos é considerado como uma tarefa árdua de se realizar, uma vez que é exigida uma adaptação dinâmica para que seja criado um agente de poker robusto. Esta tese irá focar-se no desenvolvimento de um agente de poker capaz de jogar contra jogadores humanos e alcançar a adaptação dinâmica necessária para superar alguns jogadores humanos de poker online. Algo que será possível usando um conjunto de informações sobre cada jogador que o agente enfrenta. Utilizando como auxílio o Holdem Manager, uma ferramenta que regista mãos jogadas em salas de poker online, é possível obter estatísticas sobre todos os jogadores que o agente enfrenta nas mesas. O agente é capaz de explorar algumas destas estatísticas de maneira que possa decidir melhor sobre a acção a tomar. Alguns factores como quão agressivo é um adversário, a posição ocupada na mesa, quantos jogadores estão envolvidos, quanto dinheiro está em causa, e o par de cartas que o agente recebe são uma pequena porção do conjunto de informações utilizadas na determinação do comportamento do agente. Este agente foi desenvolvido baseando-se numa estratégia "short stack", e modelando adversários com o auxílio do conjunto de informações reunido através do Holdem Manager. Pela primeira vez na literatura do Computer Poker, são apresentados resultados de jogos de poker online, num ambiente controlado, contra jogadores humanos sem estes saberem que estão em jogo contra um agente. O agente é capaz de jogar poker online ao vivo contra jogadores humanos, e apresenta um pequeno lucro na vertente Texas Hold'em em micro limites6 de apostas, nomeadamente 0.01 e 0.02 cêntimos.Games of incomplete information, such as poker, are a continuous source of research and study in the area of artificial intelligence. Poker presents challenging problems such as opponent modeling, risk management and bluff detection. The development of agents capable of probabilistic calculations considering those problems is considered to be difficult to achieve, since dynamic adaption is required in order to create a robust computer poker player. This thesis focuses on the development of a poker agent able to play against human players and aiming to achieve the dynamic adaptation needed to beat some human players online. This will be achieved by using some sets of information about each player the agent plays against. Using Holdem Manager, a tool that registers the hands played in an online poker room; it is possible to obtain statistics about every player the agent is playing against. The agent is able to explore some of these statistics so that it can better decide on which action to take. Some factors like how aggressive an opponent is, the position held at the table, how many players are involved, how much money is involved, and the hand dealt to the agent are a few portions of the information sets used to compute the agent's behavior. This agent was developed based on a short-stack strategy, and through the use of the sets of information provided by the Holdem Manager. For the first time in the Computer Poker literature, results on online Poker agent games versus human players in a controlled environment are presented, and without the players being aware their opponent was a computer agent. The agent is able to play live online poker versus human players, and presents a small profit in the No-Limit Texas Hold'em poker game at micro stakes, namely 0.02 and 0.01 cents

    Machine learning applied to the context of Poker

    Get PDF
    A combinação de princípios da teoria de jogo e metodologias de machine learning aplicados ao contexto de formular estratégias ótimas para jogos está a angariar interesse por parte de uma porção crescentemente significativa da comunidade científica, tornando-se o jogo do Poker num candidato de estudo popular devido à sua natureza de informação imperfeita. Avanços nesta área possuem vastas aplicações em cenários do mundo real, e a área de investigação de inteligência artificial demonstra que o interesse relativo a este objeto de estudo está longe de desaparecer, com investigadores do Facebook e Carnegie Mellon a apresentar, em 2019, o primeiro agente de jogo autónomo de Poker provado como ganhador num cenário com múltiplos jogadores, uma conquista relativamente à anterior especificação do estado da arte, que fora desenvolvida para jogos de apenas 2 jogadores. Este estudo pretende explorar as características de jogos estocásticos de informação imperfeita, recolhendo informação acerca dos avanços nas metodologias disponibilizados por parte de investigadores de forma a desenvolver um agente autónomo de jogo que se pretende inserir na classificação de "utility-maximizing decision-maker".The combination of game theory principles and machine learning methodologies applied to encountering optimal strategies for games is garnering interest from an increasing large portion of the scientific community, with the game of Poker being a popular study subject due to its imperfect information nature. Advancements in this area have a wide array of applications in real-world scenarios, and the field of artificial intelligent studies show that the interest regarding this object of study is yet to fade, with researchers from Facebook and Carnegie Mellon presenting, in 2019, the world’s first autonomous Poker playing agent that is proven to be profitable while confronting multiple players at a time, an achievement in relation to the previous state of the art specification, which was developed for two player games only. This study intends to explore the characteristics of stochastic games of imperfect information, gathering information regarding the advancements in methodologies made available by researchers in order to ultimately develop an autonomous agent intended to adhere to the classification of a utility-maximizing decision-maker

    Enhancing automated red teaming with Monte Carlo Tree Search

    Get PDF
    This study has investigated novel Automated Red Teaming methods that support replanning. Traditional Automated Red Teaming (ART) approaches usually use evolutionary computing methods for evolving plans using simulations. A drawback of this method is the inability to change a team’s strategy part way through a simulation. This study focussed on a Monte-Carlo Tree Search (MCTS) method in an ART environment that supports re-planning to lead to better strategy decisions and a higher average scor

    Game theoretic modeling and analysis : A co-evolutionary, agent-based approach

    Get PDF
    Ph.DDOCTOR OF PHILOSOPH

    A Deep Reinforcement Learning Neural Network Folding Proteins

    Get PDF
    Παρά τη σημαντική πρόοδο, η πρόβλεψη δομής πρωτεϊνών από την "εξ αρχής" πρωτεϊνική ακολουθία (ab initio) παραμένει ένα άλυτο πρόβλημα. Μια καλή προσέγγιση αποτελεί το ηλεκτρονικό παιχνίδι παζλ Foldit [1], το οποίο παρείχε στην επιστημονική κοινότητα αρκετά χρήσιμα αποτελέσματα, αντίστοιχα ή ακόμα και καλύτερα από τις μέχρι τώρα υπολογιστικές λύσεις [2]. Χρησιμοποιώντας το Foldit, το κοινό του WeFold [3] είχε αρκετές επιτυχημένες συμμετοχές στην κριτική αξιολόγηση τεχνικών πρόβλεψης δομής των πρωτεϊνών. Βασιζόμενοι στην πρόσφατη έκδοση του Foldit, Folditstandalone [4], εκπαιδεύσαμε ένα νευρωνικό δίκτυο βαθιάς ενισχυτικής μάθησης, το DeepFoldit, για να βελτιώσει τη βαθμολογία που δίνεται σε μια ξεδιπλωμένη πρωτεΐνη, χρησιμοποιώντας τη μέθοδο Q-learning [5] με επανάληψη εμπειρίας (experience replay). Η παρούσα διπλωματική εργασία επικεντρώνεται στη βελτίωση του μοντέλου πρόβλεψης μέσω της ρύθμισης υπερπαραμέτρων. Εξετάσαμε διάφορες υλοποιήσεις, χρησιμοποιώντας διαφορετικές αρχιτεκτονικές μοντέλων και μεταβάλλοντας τις τιμές των υπερπαραμέτρων. Καταλήξαμε σε ένα μοντέλο που επιτυγχάνει καλύτερη ακρίβεια από την αρχική υλοποίηση. Ενισχύθηκε έτσι η απόδοση με το νέο μοντέλο και βελτιώθηκε η ικανότητά του για γενίκευση. Τα αρχικά αποτελέσματα δείχνουν ότι, δεδομένης μιας σειράς μικρών ξετυλιγμένων ευθύγραμμων πρωτεϊνικών μορίων για εκπαίδευση, το DeepFoldit μαθαίνει γρήγορα τις ακολουθίες δράσης που βελτιώνουν τη βαθμολογία τόσο στα δεδομένα που χρησιμοποιήθηκαν στη διαδικασία εκπαίδευσης (training set), όσο και στις νέες δοκιμαστικές πρωτεΐνες (test set). Αυτό είναι σημαντικό καθώς η βελτίωση της βαθμολογίας του παιχνιδιού σημαίνει την επίτευξη μιας καλύτερης αναδίπλωσης, το οποίο μας φέρνει ένα βήμα πιο κοντά στην λύση. Η προσέγγισή μας συνδυάζει την έξυπνη διεπαφή του Foldit με τη δύναμη της βαθιάς ενισχυτικής μάθησης.Despite considerable progress, ab initio protein structure prediction remains unoptimised. A crowdsourcing approach is the online puzzle video game Foldit [1], that provided several useful results that matched or even outperformed algorithmically computed solutions [2]. Using Foldit, the WeFold [3] crowd had several successful participations in the Critical Assessment of Techniques for Protein Structure Prediction. Based on the recent Foldit standalone version [4], we trained a deep reinforcement neural network called DeepFoldit to improve the score assigned to an unfolded protein, using the Q-learning method [5] with experience replay. The thesis is focused on model improvement through hyperparameter tuning. We examined various implementations by examining different model architectures and changing hyperparameter values to improve the accuracy of the model. The new model’s hyper-parameters also improved its ability to generalize. Initial results, from the latest implementation, show that given a set of small unfolded training proteins, DeepFoldit learns action sequences that improve the score both on the training set and on novel test proteins. This is important as improving the game score means obtaining a better folding, taking us one step closer to the solution. Our approach combines the intuitive user interface of Foldit with the efficiency of deep reinforcement learning

    A Contextual Approach To Learning Collaborative Behavior Via Observation

    Get PDF
    This dissertation describes a novel technique to creating a simulated team of agents through observation. Simulated human teamwork can be used for a number of purposes, such as expert examples, automated teammates for training purposes and realistic opponents in games and training simulation. Current teamwork simulations require the team member behaviors be programmed into the simulation, often requiring a great deal of time and effort. None are able to observe a team at work and replicate the teamwork behaviors. Machine learning techniques for learning by observation and learning by demonstration have proven successful at observing behavior of humans or other software agents and creating a behavior function for a single agent. The research described here combines current research in teamwork simulations and learning by observation to effectively train a multi-agent system in effective team behavior. The dissertation describes the background and work by others as well as a detailed description of the learning method. A prototype built to evaluate the developed approach as well as the extensive experimentation conducted is also described

    Non-invasive wearable sensing systems for continuous health monitoring and long-term behavior modeling

    Get PDF
    Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, February 2006.Includes bibliographical references (p. 212-228).Deploying new healthcare technologies for proactive health and elder care will become a major priority over the next decade, as medical care systems worldwide become strained by the aging populations. This thesis presents LiveNet, a distributed mobile system based on low-cost commodity hardware that can be deployed for a variety of healthcare applications. LiveNet embodies a flexible infrastructure platform intended for long-term ambulatory health monitoring with real-time data streaming and context classification capabilities. Using LiveNet, we are able to continuously monitor a wide range of physiological signals together with the user's activity and context, to develop a personalized, data-rich health profile of a user over time. Most clinical sensing technologies that exist have focused on accuracy and reliability, at the expense of cost-effectiveness, burden on the patient, and portability. Future proactive health technologies, on the other hand, must be affordable, unobtrusive, and non-invasive if the general population is going to adopt them.(cont.) In this thesis, we focus on the potential of using features derived from minimally invasive physiological and contextual sensors such as motion, speech, heart rate, skin conductance, and temperature/heat flux that can be used in combination with mobile technology to create powerful context-aware systems that are transparent to the user. In many cases, these non-invasive sensing technologies can completely replace more invasive diagnostic sensing for applications in long-term monitoring, behavior and physiology trending, and real-time proactive feedback and alert systems. Non-invasive sensing technologies are particularly important in ambulatory and continuous monitoring applications, where more cumbersome sensing equipment that is typically found in medical and clinical research settings is not usable. The research in this thesis demonstrates that it is possible to use simple non-invasive physiological and contextual sensing using the LiveNet system to accurately classify a variety of physiological conditions. We demonstrate that non-invasive sensing can be correlated to a variety of important physiological and behavioral phenomenon, and thus can serve as substitutes to more invasive and unwieldy forms of medical monitoring devices while still providing a high level of diagnostic power.(cont.) From this foundation, the LiveNet system is deployed in a number of studies to quantify physiological and contextual state. First, a number of classifiers for important health and general contextual cues such as activity state and stress level are developed from basic non-invasive physiological sensing. We then demonstrate that the LiveNet system can be used to develop systems that can classify clinically significant physiological and pathological conditions and that are robust in the presence of noise, motion artifacts, and other adverse conditions found in real-world situations. This is highlighted in a cold exposure and core body temperature study in collaboration with the U.S. Army Research Institute of Environmental Medicine. In this study, we show that it is possible to develop real-time implementations of these classifiers for proactive health monitors that can provide instantaneous feedback relevant in soldier monitoring applications. This thesis also demonstrates that the LiveNet platform can be used for long-term continuous monitoring applications to study physiological trends that vary slowly with time.(cont.) In a clinical study with the Psychiatry Department at the Massachusetts General Hospital, the LiveNet platform is used to continuously monitor clinically depressed patients during their stays on an in-patient ward for treatment. We show that we can accurately correlate physiology and behavior to depression state, as well as to track changes in depression state over time through the course of treatment. This study demonstrates how long-term physiology and behavioral changes can be captured to objectively measure medical treatment and medication efficacy. In another long-term monitoring study, the LiveNet platform is used to collect data on people's everyday behavior as they go through daily life. By collecting long-term behavioral data, we demonstrate the possibility of modeling and predicting high-level behavior using simple physiologic and contextual information derived solely from ambulatory mobile sensing technology.by Michael Sung.Ph.D

    Market microstructure: the automated order book

    Get PDF
    corecore