Generating Levels and Playing Super Mario Bros. with Deep Reinforcement Learning Using various techniques for level generation and Deep Q-Networks for playing

Master's thesis in Information- and communication technology (IKT590)This thesis aims to explore the behavior of two competing reinforcement learning agents in Super Mario Bros. In video games, PCG can be used to assist human game designers by generating a particular aspect of the game. A human game designer can use generated game content as inspiration to build further upon, which saves time and resources. Much research has been conducted on AI in video games, including AI for playing Super Mario Bros. Additionally, there exists a research field focused on PCG for video games, which includes generation of Super Mario Bros. levels. In this thesis, the two fields of research are combined to form a GAN-inspired system of two competing AI agents. One agent is controlling Mario, and this agent represents the discriminator. The other agent generates the level Mario is playing, and represents the generator. In an ordinary GAN system, the generator is attempting to mimic a database containing real data, while the discriminator attempts to distinguish real data samples from the generated data samples. The Mario agent utilizes a DQN algorithm for learning to navigate levels, while the level generator uses a DQN-based algorithm with different types of neural networks. The DQN algorithm utilizes neural networks to predict the expected future reward for each possible action. The expected future rewards are denoted as Q-values. The results show that the generator is capable of generating content better than random when the generator model takes a sequence of tiles as input and produces a sequence of predictions of Q-values as output

Observation of the 4π\pi-periodic Josephson effect in indium arsenide nanowires

Quantum computation by non-Abelian Majorana zero modes (MZMs) offers an approach to achieve fault tolerance by encoding quantum information in the non-local charge parity states of semiconductor nanowire networks in the topological superconductor regime. Thus far, experimental studies of MZMs chiefly relied on single electron tunneling measurements which leads to decoherence of the quantum information stored in the MZM. As a next step towards topological quantum computation, charge parity conserving experiments based on the Josephson effect are required, which can also help exclude suggested non-topological origins of the zero bias conductance anomaly. Here we report the direct measurement of the Josephson radiation frequency in InAs nanowires with epitaxial aluminium shells. For the first time, we observe the $4\pi$-periodic Josephson effect above a magnetic field of $\approx 200\,$mT, consistent with the estimated and measured topological phase transition of similar devices.Comment: Published version. Supplementary Information is available as ancillary file, raw data and calculations can be downloaded from http://dx.doi.org/10.4121/uuid:1f936840-5bc2-40ca-8c32-1797c12cacb

Electrical manipulation of spin states in a single electrostatically gated transition-metal complex

We demonstrate an electrically controlled high-spin (S=5/2) to low-spin (S=1/2) transition in a three-terminal device incorporating a single Mn2+ ion coordinated by two terpyridine ligands. By adjusting the gate-voltage we reduce the terpyridine moiety and thereby strengthen the ligand-field on the Mn-atom. Adding a single electron thus stabilizes the low-spin configuration and the corresponding sequential tunnelling current is suppressed by spin-blockade. From low-temperature inelastic cotunneling spectroscopy, we infer the magnetic excitation spectrum of the molecule and uncover also a strongly gate-dependent singlet-triplet splitting on the low-spin side. The measured bias-spectroscopy is shown to be consistent with an exact diagonalization of the Mn-complex, and an interpretation of the data is given in terms of a simplified effective model.Comment: Will appear soon in Nanoletter

Effects of Anacetrapib in Patients with Atherosclerotic Vascular Disease

BACKGROUND: Patients with atherosclerotic vascular disease remain at high risk for cardiovascular events despite effective statin-based treatment of low-density lipoprotein (LDL) cholesterol levels. The inhibition of cholesteryl ester transfer protein (CETP) by anacetrapib reduces LDL cholesterol levels and increases high-density lipoprotein (HDL) cholesterol levels. However, trials of other CETP inhibitors have shown neutral or adverse effects on cardiovascular outcomes. METHODS: We conducted a randomized, double-blind, placebo-controlled trial involving 30,449 adults with atherosclerotic vascular disease who were receiving intensive atorvastatin therapy and who had a mean LDL cholesterol level of 61 mg per deciliter (1.58 mmol per liter), a mean non-HDL cholesterol level of 92 mg per deciliter (2.38 mmol per liter), and a mean HDL cholesterol level of 40 mg per deciliter (1.03 mmol per liter). The patients were assigned to receive either 100 mg of anacetrapib once daily (15,225 patients) or matching placebo (15,224 patients). The primary outcome was the first major coronary event, a composite of coronary death, myocardial infarction, or coronary revascularization. RESULTS: During the median follow-up period of 4.1 years, the primary outcome occurred in significantly fewer patients in the anacetrapib group than in the placebo group (1640 of 15,225 patients [10.8%] vs. 1803 of 15,224 patients [11.8%]; rate ratio, 0.91; 95% confidence interval, 0.85 to 0.97; P=0.004). The relative difference in risk was similar across multiple prespecified subgroups. At the trial midpoint, the mean level of HDL cholesterol was higher by 43 mg per deciliter (1.12 mmol per liter) in the anacetrapib group than in the placebo group (a relative difference of 104%), and the mean level of non-HDL cholesterol was lower by 17 mg per deciliter (0.44 mmol per liter), a relative difference of -18%. There were no significant between-group differences in the risk of death, cancer, or other serious adverse events. CONCLUSIONS: Among patients with atherosclerotic vascular disease who were receiving intensive statin therapy, the use of anacetrapib resulted in a lower incidence of major coronary events than the use of placebo. (Funded by Merck and others; Current Controlled Trials number, ISRCTN48678192 ; ClinicalTrials.gov number, NCT01252953 ; and EudraCT number, 2010-023467-18 .)

Generating Levels and Playing Super Mario Bros. with Deep Reinforcement Learning Using various techniques for level generation and Deep Q-Networks for playing

This thesis aims to explore the behavior of two competing reinforcement learning agents in Super Mario Bros. In video games, PCG can be used to assist human game designers by generating a particular aspect of the game. A human game designer can use generated game content as inspiration to build further upon, which saves time and resources. Much research has been conducted on AI in video games, including AI for playing Super Mario Bros. Additionally, there exists a research field focused on PCG for video games, which includes generation of Super Mario Bros. levels. In this thesis, the two fields of research are combined to form a GAN-inspired system of two competing AI agents. One agent is controlling Mario, and this agent represents the discriminator. The other agent generates the level Mario is playing, and represents the generator. In an ordinary GAN system, the generator is attempting to mimic a database containing real data, while the discriminator attempts to distinguish real data samples from the generated data samples. The Mario agent utilizes a DQN algorithm for learning to navigate levels, while the level generator uses a DQN-based algorithm with different types of neural networks. The DQN algorithm utilizes neural networks to predict the expected future reward for each possible action. The expected future rewards are denoted as Q-values. The results show that the generator is capable of generating content better than random when the generator model takes a sequence of tiles as input and produces a sequence of predictions of Q-values as output

Observation of the 4π-periodic Josephson effect in InAs nanowires

Raw data and code that belong to the paper "Observation of the 4π-periodic Josephson effect in InAs nanowires". For usage instructions, see readme.txt in the experiment and theory folder, respectively

Tracking data highlight the importance of human-induced mortality for large migratory birds at a flyway scale

Human-induced direct mortality affects huge numbers of birds each year, threatening hundreds of species worldwide. Tracking technologies can be an important tool to investigate temporal and spatial patterns of bird mortality as well as their drivers. We compiled 1704 mortality records from tracking studies across the African-Eurasian flyway for 45 species, including raptors, storks, and cranes, covering the period from 2003 to 2021. Our results show a higher frequency of human-induced causes of mortality than natural causes across taxonomic groups, geographical areas, and age classes. Moreover, we found that the frequency of human-induced mortality remained stable over the study period. From the human-induced mortality events with a known cause (n = 637), three main causes were identified: electrocution (40.5 %), illegal killing (21.7 %), and poisoning (16.3 %). Additionally, combined energy infrastructure-related mortality (i.e., electrocution, power line collision, and wind-farm collision) represented 49 % of all human-induced mortality events. Using a random forest model, the main predictors of human-induced mortality were found to be taxonomic group, geographic location (latitude and longitude), and human footprint index value at the location of mortality. Despite conservation efforts, human drivers of bird mortality in the African-Eurasian flyway do not appear to have declined over the last 15 years for the studied group of species. Results suggest that stronger conservation actions to address these threats across the flyway can reduce their impacts on species. In particular, projected future development of energy infrastructure is a representative example where application of planning, operation, and mitigation measures can enhance bird conservation.publishedVersio