Macro action selection with deep reinforcement learning in StarCraft

StarCraft (SC) is one of the most popular and successful Real Time Strategy (RTS) games. In recent years, SC is also widely accepted as a challenging testbed for AI research because of its enormous state space, partially observed information, multi-agent collaboration, and so on. With the help of annual AIIDE and CIG competitions, a growing number of SC bots are proposed and continuously improved. However, a large gap remains between the top-level bot and the professional human player. One vital reason is that current SC bots mainly rely on predefined rules to select macro actions during their games. These rules are not scalable and efficient enough to cope with the enormous yet partially observed state space in the game. In this paper, we propose a deep reinforcement learning (DRL) framework to improve the selection of macro actions. Our framework is based on the combination of the Ape-X DQN and the Long-Short-Term-Memory (LSTM). We use this framework to build our bot, named as LastOrder. Our evaluation, based on training against all bots from the AIIDE 2017 StarCraft AI competition set, shows that LastOrder achieves an 83% winning rate, outperforming 26 bots in total 28 entrants

Mechanical Characterizations of Oxidizing Steel Slag Soil and Application

In this paper, the mechanical properties and engineering applicationof electric furnace (EAF) slag mixed soil are investigated.The samples of steel slag are taken from a steel manufacturingcompany in Huangshi, a city of China. The mixed soilwas firstly prepared by mixing the steel slag and clay mixturein different proportions. The optimal moisture content for mixingthe soil is investigated from the experiment through directshear test. Based on three axial compression tests, the optimumsteel slag ratio is determined. Finally, the mechanical propertiesof steel slag mixed soil are tested in a practical engineeringproblem through a numerical simulation. The steel slag mixedsoil is used to replace the original soil of the embankment andcompared with that of the original one. The comparison studyshows that the method proposed in this paper is simple andeffective. Moreover, from the practical problem analysis, theoptimal utilization of electric furnace slag can be achieved

Structural Holes in Directed Fuzzy Social Networks

The structural holes have been a key issue in fuzzy social network analysis. For undirected fuzzy social networks where edges are just present or absent undirected fuzzy relation and have no more information attached, many structural holes measures have been presented, such as key fuzzy structural holes, general fuzzy structural holes, strong fuzzy structural holes, and weak fuzzy structural holes. There has been a growing need to design structural holes measures for directed fuzzy social networks, because directed fuzzy social networks where edges are attached by directed fuzzy relation would contain rich information. In this paper, we extend structural holes theory to directed fuzzy social network and propose the algorithm of unidirectional fuzzy structural holes and bidirectional fuzzy structural holes, which unveil more structural information of directed fuzzy social networks. Furthermore, we investigate the validness of the algorithm by illustrating this method to a case called G-Y Research Team and obtain reliable results, which provide strong evidences of the new measure’s utility

Macro action selection with deep reinforcement learning in StarCraft

StarCraft (SC) is one of the most popular and successful Real Time Strategy (RTS) games. In recent years, SC is also considered as a testbed for AI research, due to its enormous state space, hidden information, multi-agent collaboration and so on. Thanks to the annual AIIDE and CIG competitions, a growing number of bots are proposed and being continuously improved. However, a big gap still remains between the top bot and the professional human players. One vital reason is that current bots mainly rely on predefined rules to perform macro actions. These rules are not scalable and efficient enough to cope with the large but partially observed macro state space in SC. In this paper, we propose a DRL based framework to do macro action selection. Our framework combines the reinforcement learning approach Ape-X DQN with Long-Short-Term-Memory (LSTM) to improve the macro action selection in bot. We evaluate our bot, named as LastOrder, on the AIIDE 2017 StarCraft AI competition bots set. Our bot achieves overall 83% win-rate, outperforming 26 bots in total 28 entrants

Exposing the Causal Effect of Body Mass Index on the Risk of Type 2 Diabetes Mellitus: A Mendelian Randomization Study

Introduction: High body mass index (BMI) is a positive associated phenotype of type 2 diabetes mellitus (T2DM). Abundant studies have observed this from a clinical perspective. Since the rapid increase in a large number of genetic variants from the genome-wide association studies (GWAS), common SNPs of BMI and T2DM were identified as the genetic basis for understanding their associations. Currently, their causality is beginning to blur.Materials and Methods: To classify it, a Mendelian randomisation (MR), using genetic instrumental variables (IVs) to explore the causality of intermediate phenotype and disease, was utilized here to test the effect of BMI on the risk of T2DM. In this article, MR was carried out on GWAS data using 52 independent BMI SNPs as IVs. The pooled odds ratio (OR) of these SNPs was calculated using inverse-variance weighted method for the assessment of 5 kg/m2 higher BMI on the risk of T2DM. The leave-one-out validation was conducted to identify the effect of individual SNPs. MR-Egger regression was utilized to detect potential pleiotropic bias of variants.Results: We obtained the high OR (1.470; 95% CI 1.170 to 1.847; P = 0.001), low intercept (0.004, P = 0.661), and small fluctuation of ORs {from -0.039 [(1.412 – 1.470) / 1.470)] to 0.075 [(1.568– 1.470) / 1.470)] in leave-one-out validation.Conclusion: We validate the causal effect of high BMI on the risk of T2DM. The low intercept shows no pleiotropic bias of IVs. The small alterations of ORs activated by removing individual SNPs showed no single SNP drives our estimate

Virulence Determinants Are Required for Brain Abscess Formation Through Staphylococcus aureus Infection and Are Potential Targets of Antivirulence Factor Therapy

Bacterial brain abscesses (BAs) are difficult to treat with conventional antibiotics. Thus, the development of alternative therapeutic strategies for BAs is of high priority. Identifying the virulence determinants that contribute to BA formation induced by Staphylococcus aureus would improve the effectiveness of interventions for this disease. In this study, RT-qPCR was performed to compare the expression levels of 42 putative virulence determinants of S. aureus strains Newman and XQ during murine BA formation, ear colonization, and bacteremia. The alterations in the expression levels of 23 genes were further confirmed through specific TaqMan RT-qPCR. Eleven S. aureus genes that persistently upregulated expression levels during BA infection were identified, and their functions in BA formation were confirmed through isogenic mutant experiments. Bacterial loads and BA volumes in mice infected with isdA, isdC, lgt, hla, or spa deletion mutants and the hla/spa double mutant strain were lower than those in mice infected with the wild-type Newman strain. The therapeutic application of monoclonal antibodies against Hla and SpA decreased bacterial loads and BA volume in mice infected with Newman. This study provides insights into the virulence determinants that contribute to staphylococcal BA formation and a paradigm for antivirulence factor therapy against S. aureus infections