Exploration via Epistemic Value Estimation

How to efficiently explore in reinforcement learning is an open problem. Many exploration algorithms employ the epistemic uncertainty of their own value predictions - for instance to compute an exploration bonus or upper confidence bound. Unfortunately the required uncertainty is difficult to estimate in general with function approximation. We propose epistemic value estimation (EVE): a recipe that is compatible with sequential decision making and with neural network function approximators. It equips agents with a tractable posterior over all their parameters from which epistemic value uncertainty can be computed efficiently. We use the recipe to derive an epistemic Q-Learning agent and observe competitive performance on a series of benchmarks. Experiments confirm that the EVE recipe facilitates efficient exploration in hard exploration tasks

Unraveling the Effects of Acute Inflammation on Pharmacokinetics: A Model-Based Analysis Focusing on Renal Glomerular Filtration Rate and Cytochrome P450 3A4-Mediated Metabolism

Background and Objectives Acute inflammation caused by infections or sepsis can impact pharmacokinetics. We used a model-based analysis to evaluate the effect of acute inflammation as represented by interleukin-6 (IL-6) levels on drug clearance, focusing on renal glomerular filtration rate (GFR) and cytochrome P450 3A4 (CYP3A4)-mediated metabolism. Methods A physiologically based model incorporating renal and hepatic drug clearance was implemented. Functions correlating IL-6 levels with GFR and in vitro CYP3A4 activity were derived and incorporated into the modeling framework. We then simulated treatment scenarios for hypothetical drugs by varying the IL-6 levels, the contribution of renal and hepatic drug clearance, and protein binding. The relative change in observed area under the concentration-time curve (AUC) was computed for these scenarios. Results Inflammation showed opposite effects on drug exposure for drugs eliminated via the liver and kidney, with the effect of inflammation being inversely proportional to the extraction ratio (ER). For renally cleared drugs, the relative decrease in AUC was close to 30% during severe inflammation. For CYP3A4 substrates, the relative increase in AUC could exceed 50% for low-ER drugs. Finally, the impact of inflammation-induced changes in drug clearance is smaller for drugs with a larger unbound fraction. Conclusion This analysis demonstrates differences in the impact of inflammation on drug clearance for different drug types. The effects of inflammation status on pharmacokinetics may explain the inter-individual variability in pharmacokinetics in critically ill patients. The proposed model-based analysis may be used to further evaluate the effect of inflammation, i.e., by incorporating the effect of inflammation on other drug-metabolizing enzymes or physiological processes

Increase in circulating Foxp3+CD4+CD25high regulatory T cells in nasopharyngeal carcinoma patients

Nasopharyngeal carcinoma (NPC) is an Epstein–Barr virus-associated disease with high prevalence in Southern Chinese. Using multiparametric flow cytometry, we identified significant expansions of circulating naïve and memory CD4+CD25high T cells in 56 NPC patients compared with healthy age- and sex-matched controls. These were regulatory T cells (Treg), as they overexpressed Foxp3 and GITR, and demonstrated enhanced suppressive activities against autologous CD4+CD25− T-cell proliferation in functional studies on five patients. Abundant intraepithelial infiltrations of Treg with very high levels of Foxp3 expression and absence of CCR7 expression were also detected in five primary tumours. Our current study is the first to demonstrate an expansion of functional Treg in the circulation of NPC patients and the presence of infiltrating Treg in the tumour microenvironment. As Treg may play an important role in suppressing antitumour immunity, our findings provide critical insights for clinical management of NPC

On the importance of the heterogeneity assumption in the characterization of reservoir geomechanical properties

The geomechanical analysis of a highly compartmentalized reservoir is performed to simulate the seafloor subsidence due to gas production. The available observations over the hydrocarbon reservoir consist of bathymetric surveys carried out before and at the end of a 10-yr production life. The main goal is the calibration of the reservoir compressibility cM, that is, the main geomechanical parameter controlling the surface response. Two conceptual models are considered: in one (i) cM varies only with the depth and the vertical effective stress (heterogeneity due to lithostratigraphic variability); in another (ii) cM varies also in the horizontal plane, that is, it is spatially distributed within the reservoir stratigraphic units. The latter hypothesis accounts for a possible partitioning of the reservoir due to the presence of sealing faults and thrusts that suggests the idea of a block heterogeneous system with the number of reservoir blocks equal to the number of uncertain parameters. The method applied here relies on an ensemble-based data assimilation (DA) algorithm (i.e. the ensemble smoother, ES), which incorporates the information from the bathymetric measurements into the geomechanical model response to infer and reduce the uncertainty of the parameter cM. The outcome from conceptual model (i) indicates that DA is effective in reducing the cM uncertainty. However, the maximum settlement still remains underestimated, while the areal extent of the subsidence bowl is overestimated. We demonstrate that the selection of the heterogeneous conceptual model (ii) allows to reproduce much better the observations thus removing a clear bias of the model structure. DA allows significantly reducing the cM uncertainty in the five blocks (out of the seven) characterized by large volume and large pressure decline. Conversely, the assimilation of land displacements only partially constrains the prior cM uncertainty in the reservoir blocks marginally contributing to the cumulative seafloor subsidence, that is, blocks with low pressure

Biomarker-guided individualization of antibiotic therapy

Treatment failure of antibiotic therapy due to insufficient efficacy or occurrence of toxicity is a major clinical challenge, and is expected to become even more urgent with the global rise of antibiotic resistance. Strategies to optimize treatment in individual patients are therefore of crucial importance. Currently, therapeutic drug monitoring plays an important role in optimizing antibiotic exposure to reduce treatment failure and toxicity. Biomarker-based strategies may be a powerful tool to further quantify and monitor antibiotic treatment response, and reduce variation in treatment response between patients. Host response biomarkers, such as CRP, procalcitonin, IL-6, and presepsin, could potentially carry significant information to be utilized for treatment individualization. To achieve this, the complex interactions among immune system, pathogen, drug, and biomarker need to be better understood and characterized. The purpose of this tutorial is to discuss the use and evidence of currently available biomarker-based approaches to inform antibiotic treatment. To this end, we also included a discussion on how treatment response biomarker data from preclinical, healthy volunteer, and patient-based studies can be further characterized using pharmacometric and system pharmacology based modeling approaches. As an illustrative example of how such modeling strategies can be used, we describe a case study in which we quantitatively characterize procalcitonin dynamics in relation to antibiotic treatments in patients with sepsis.Pharmacolog

Structure-based prediction of anti-infective drug concentrations in the human lung epithelial lining fluid

Pharmacolog