Technical Challenges of Deploying Reinforcement Learning Agents for Game Testing in AAA Games

Going from research to production, especially for large and complex software systems, is fundamentally a hard problem. In large-scale game production, one of the main reasons is that the development environment can be very different from the final product. In this technical paper we describe an effort to add an experimental reinforcement learning system to an existing automated game testing solution based on scripted bots in order to increase its capacity. We report on how this reinforcement learning system was integrated with the aim to increase test coverage similar to [1] in a set of AAA games including Battlefield 2042 and Dead Space (2023). The aim of this technical paper is to show a use-case of leveraging reinforcement learning in game production and cover some of the largest time sinks anyone who wants to make the same journey for their game may encounter. Furthermore, to help the game industry to adopt this technology faster, we propose a few research directions that we believe will be valuable and necessary for making machine learning, and especially reinforcement learning, an effective tool in game production.Comment: 8 pages, 5 figure

Association between Plasma Homocysteine Levels and Neuronal Injury in HIV Infection

OBJECTIVE: To investigate the role of homocysteine in neuronal injury in HIV infection. METHODS: Using a cross-sectional design and archived samples, we compared concentrations of plasma homocysteine and cerebrospinal fluid (CSF) neurofilament light protein (NFL), a sensitive marker of neuronal injury, in 83 HIV-1-infected subjects without antiretroviral treatment. We also analyzed plasma vitamin B12, serum folate, CSF, and plasma HIV RNA, the immune activation marker neopterin in CSF and serum, and albumin ratio as a marker of blood-brain barrier integrity. Twenty-two subjects provided a second sample median of 12.5 months after antiretroviral treatment initiation. RESULTS: A significant correlation was found between plasma homocysteine and CSF NFL concentrations in untreated individuals (r = 0.52, p < 0.0001). As expected, there was a significant inverse correlation between homocysteine and B12 (r = –0.41, p < 0.001) and folate (r = –0.40, p = < 0.001) levels. In a multiple linear regression analysis homocysteine stood out as an independent predictor of CSF NFL in HIV-1-infected individuals. The correlation of plasma homocysteine and CSF NFL was also present in the group receiving antiretroviral therapy (r = 0.51, p = 0.016). CONCLUSION: A correlation between plasma homocysteine and axonal injury, as measured by CSF NFL, was found in both untreated and treated HIV. While this study is not able to prove a causa

Serum neopterin levels in relation to mild and severe COVID-19

Background: The COVID-19 pandemic, caused by the coronavirus SARS-CoV-2, is rapidly spreading worldwide. There is limited information about prognostic markers that could help clinicians to identify COVID-19 patients with a poor prognosis. Serum levels of the immune activation marker neopterin has shown to be of prognostic value in patients with SARS. The aim of this study was to investigate whether serum neopterin is associated with the severity of COVID-19. Methods: We included 34 patients with confirmed COVID-19 between March 3 and March 30, 2020. Fifteen patients had mild disease and did not require hospitalization, whereas 19 patients developed severe COVID-19 requiring intensive care. Concentrations of serum neopterin, tryptophan, and kynurenine were measured at and repeatedly after inclusion. Results: We found a more than two-fold higher mean concentration of neopterin in severely ill patients (mean value 42.0 nmol/L (SD 18.2)) compared to patients with mild symptoms (16.9 nmol/L (SD 11.0)). All of the severe cases had elevated neopterin concentrations (&gt; 9.1 nmol/L) at the initial sampling with values ranging from 17.2 to 86.7 nmol/L. In comparison, 10 of 15 patients with mild disease had neopterin levels above 9.1 nmol/L, with concentrations in the range from 4.9 to 31.6 nmol/L. Neopterin levels gradually decreased during the course of COVID-19, but severe cases maintained elevated levels for a longer period. Moreover, lower levels of tryptophan and higher levels of kynurenine, indicating an increased tryptophan catabolism, were seen in the group with severe cases. Conclusions: In conclusion, we found that serum neopterin levels are associated with the severity of COVID-19. Our findings suggest that neopterin could be used as a prognostic marker, but further studies are needed to elucidate how it can be used in the clinic

Switching from a regimen containing abacavir/lamivudine or emtricitabine/tenofovir disoproxil fumarate to emtricitabine/tenofovir alafenamide fumarate does not affect central nervous system HIV-1 infection

Background: Despite suppressive antiretroviral therapy (ART), many HIV-infected individuals have low-level persistent immune activation in the central nervous system (CNS). There have been concerns regarding the CNS efficacy of tenofovir alafenamide fumarate (TAF) because of its low cerebrospinal fluid (CSF) concentrations and because it is a substrate of the active efflux transporter P-glycoprotein. Our aim was to investigate whether switching from emtricitabine (FTC)/tenofovir disoproxil fumarate (TDF) or abacavir (ABC)/lamivudine (3TC) to FTC/TAF would lead to changes in residual intrathecal immune activation, viral load, or neurocognitive function. Methods: Twenty HIV-1-infected neuro-asymptomatic adults (11 on ABC/3TC and 9 on FTC/TDF) were included in this prospective study. At baseline, all participants changed their nucleoside analogues to FTC/TAF without any other changes in their ART regimen. We performed lumbar punctures, venipunctures, and neurocognitive testing at baseline and after three and 12 months. Results: During follow-up, there were no significant changes in CSF or plasma HIV RNA, CSF neopterin, CSF β2-microglobulin, IgG index, albumin ratio, CSF NFL, or neurocognitive function in assessed by Cogstate in any of the groups. Conclusion: This small pilot study indicates that switching to FTC/TAF from ABC/3TC or FTC/TDF has neither a positive, nor a negative effect on the HIV infection in the CNS

Optical spectra obtained from amorphous films of rubrene: Evidence for predominance of twisted isomer

In order to investigate the optical properties of rubrene we study the vibronic progression of the first absorption band (lowest \pi -> \pi^* transition). We analyze the dielectric function of rubrene in solution and thin films using the displaced harmonic oscillator model and derive all relevant parameters of the vibronic progression. The findings are supplemented by density functional calculations using B3LYP hybrid functionals. Our theoretical results for the molecule in two different conformations, i.e. with a twisted or planar tetracene backbone, are in very good agreement with the experimental data obtained for rubrene in solution and thin films. Moreover, a simulation based on the monomer spectrum and the calculated transition energies of the two conformations indicates that the thin film spectrum of rubrene is dominated by the twisted isomer.Comment: 7 pages, 6 figure

Effect of antiretroviral treatment on blood-brain barrier integrity in HIV-1 infection

Background Blood-brain barrier (BBB) injury is prevalent in patients with HIV-associated dementia (HAD) and is a frequent feature of HIV encephalitis. Signs of BBB damage are also sometimes found in neuroasymptomatic HIV-infected individuals without antiretroviral therapy (ART). The aim of this study was to investigate the integrity of the BBB before and after initiation of ART in both neuroasymptomatic HIV infection and in patients with HAD. Methods We determined BBB integrity by measuring cerebrospinal fluid (CSF)/plasma albumin ratios in archived CSF samples prior to and after initiation of ART in longitudinally-followed neuroasymptomatic HIV-1-infected individuals and patients with HAD. We also analyzed HIV RNA in blood and CSF, IgG Index, CSF WBC counts, and CSF concentrations of beta 2-micoglobulin, neopterin, and neurofilament light chain protein (NfL). Results We included 159 HIV-infected participants; 82 neuroasymptomatic individuals and 77 with HAD. All neuroasymptomatic individuals (82/82), and 10/77 individuals with HAD, were longitudinally followed with a median (interquartile range, IQR) follow-up of 758 (230-1752) days for the neuroasymptomatic individuals, and a median (IQR) follow-up of 241 (50-994) days for the individuals with HAD. Twelve percent (10/82) of the neuroasymptomatic individuals and 80% (8/10) of the longitudinally-followed individuals with HAD had elevated albumin ratios at baseline. At the last follow-up, 9% (7/82) of the neuroasymptomatic individuals and 20% (2/10) of the individuals with HAD had elevated albumin ratios. ART significantly decreased albumin ratios in both neuroasymptomatic individuals and in patients with HAD. Conclusion These findings indicate that ART improves and possibly normalizes BBB integrity in both neuroasymptomatic HIV-infected individuals and in patients with HAD

Proteomic blood profiling in mild, severe and critical COVID-19 patients

The recent SARS-CoV-2 pandemic manifests itself as a mild respiratory tract infection in most individuals, leading to COVID-19 disease. However, in some infected individuals, this can progress to severe pneumonia and acute respiratory distress syndrome (ARDS), leading to multi-organ failure and death. This study explores the proteomic differences between mild, severe, and critical COVID-19 positive patients to further understand the disease progression, identify proteins associated with disease severity, and identify potential therapeutic targets. Blood protein profiling was performed on 59 COVID-19 mild (n = 26), severe (n = 9) or critical (n = 24) cases and 28 controls using the OLINK inflammation, autoimmune, cardiovascular and neurology panels. Differential expression analysis was performed within and between disease groups to generate nine different analyses. From the 368 proteins measured per individual, more than 75% were observed to be significantly perturbed in COVID-19 cases. Six proteins (IL6, CKAP4, Gal-9, IL-1ra, LILRB4 and PD-L1) were identified to be associated with disease severity. The results have been made readily available through an interactive web-based application for instant data exploration and visualization, and can be accessed at https://phidatalab-shiny.rosalind.kcl.ac.uk/COVID19/. Our results demonstrate that dynamic changes in blood proteins associated with disease severity can potentially be used as early biomarkers to monitor disease severity in COVID-19 and serve as potential therapeutic targets

Cerebrospinal fluid neopterin: an informative biomarker of central nervous system immune activation in HIV-1 infection

HIV-1 invades the central nervous system (CNS) in the context of acute infection, persists thereafter in the absence of treatment, and leads to chronic intrathecal immunoactivation that can be measured by the macrophage activation marker, neopterin, in cerebrospinal fluid (CSF). In this review we describe our experience with CSF neopterin measurements in 382 untreated HIV-infected patients across the spectrum of immunosuppression and HIV-related neurological diseases, in 73 untreated AIDS patients with opportunistic CNS infections, and in 233 treated patients

The effect of vitamin B supplementation on neuronal injury in people living with HIV: a randomized controlled trial

Effective antiretroviral therapy has radically changed the course of the HIV pandemic. However, despite efficient therapy, milder forms of neurocognitive symptoms are still present in people living with HIV. Plasma homocysteine is a marker of vitamin B deficiency and has been associated with cognitive impairment. People living with HIV have higher homocysteine concentrations than HIV-negative controls, and we have previously found an association between plasma homocysteine concentration and CSF concentration of neurofilament light protein, a sensitive marker for ongoing neuronal injury in HIV. This prompted us to perform this randomized controlled trial, to evaluate the effect of vitamin B supplementation on neuronal injury in a cohort of people living with HIV on stable antiretroviral therapy. At the Department of Infectious Diseases at Sahlgrenska University Hospital in Gothenburg, Sweden, 124 virally suppressed people living with HIV were screened to determine eligibility for this study. Sixty-one fulfilled the inclusion criteria by having plasma homocysteine levels at or above 12 mu mol/l. They were randomized (1:1) to either active treatment (with cyanocobalamin 0.5 mg, folic acid 0.8 mg and pyridoxine 3.0 mg) q.d. or to a control arm with a cross over to active treatment after 12 months. Cognitive function was measured repeatedly during the trial, which ran for 24 months. We found a significant correlation between plasma neurofilament light protein and plasma homocysteine at screening (n = 124, r = 0.35, P &lt; 0.0001). Plasma homocysteine levels decreased by 35% from a geometric mean of 15.7 mu mol/l (95% confidence interval 14.7-16.7) to 10.3 mu mol/l (95% confidence interval 9.3-11.3) in the active treatment arm between baseline and Month 12. No significant change was detected in the control arm during the same time period [geometric mean 15.2 (95% confidence interval 14.3-16.2) versus geometric mean 16.5 mu mol/l (95% confidence interval 14.7-18.6)]. A significant difference in change in plasma homocysteine levels was seen between arms at 12 months [-40% (95% confidence interval -48 to -30%), P &lt; 0.001]. However, no difference between arms was seen in either plasma neurofilament light protein levels [-6.5% (-20 to 9%), P = 0.39], or cognitive measures [-0.08 (-0.33 to 0.17), P = 0.53]. Our results do not support a vitamin B-dependent cause of the correlation between neurofilament light protein and homocysteine. Additional studies are needed to further elucidate this matter. Tyrberg et al. report the results of a randomized controlled trial investigating the effect of vitamin B supplementation on neuronal injury in people living with HIV with effective antiretroviral therapy. Supplementation decreased levels of homocysteine but not neuronal injury measured by neurofilament light protein