Quantum Natural Policy Gradients: Towards Sample-Efficient Reinforcement Learning

Reinforcement learning is a growing field in AI with a lot of potential. Intelligent behavior is learned automatically through trial and error in interaction with the environment. However, this learning process is often costly. Using variational quantum circuits as function approximators can reduce this cost. In order to implement this, we propose the quantum natural policy gradient (QNPG) algorithm -- a second-order gradient-based routine that takes advantage of an efficient approximation of the quantum Fisher information matrix. We experimentally demonstrate that QNPG outperforms first-order based training on Contextual Bandits environments regarding convergence speed and stability and thereby reduces the sample complexity. Furthermore, we provide evidence for the practical feasibility of our approach by training on a 12-qubit hardware device.Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible. 7 pages, 5 figures, 1 tabl

Quantum Policy Gradient Algorithm with Optimized Action Decoding

Quantum machine learning implemented by variational quantum circuits (VQCs) is considered a promising concept for the noisy intermediate-scale quantum computing era. Focusing on applications in quantum reinforcement learning, we propose a specific action decoding procedure for a quantum policy gradient approach. We introduce a novel quality measure that enables us to optimize the classical post-processing required for action selection, inspired by local and global quantum measurements. The resulting algorithm demonstrates a significant performance improvement in several benchmark environments. With this technique, we successfully execute a full training routine on a 5-qubit hardware device. Our method introduces only negligible classical overhead and has the potential to improve VQC-based algorithms beyond the field of quantum reinforcement learning.Comment: Accepted to the 40th International Conference on Machine Learning (ICML 2023), Honolulu, Hawaii, USA. 22 pages, 10 figures, 3 table

Glutamate prevents intestinal atrophy via luminal nutrient sensing in a mouse model of total parenteral nutrition

Small intestine luminal nutrient sensing may be crucial for modulating physiological functions. However, its mechanism of action is incompletely understood. We used a model of enteral nutrient deprivation, or total parenteral nutrition (TPN), resulting in intestinal mucosal atrophy and decreased epithelial barrier function (EBF). We examined how a single amino acid, glutamate (GLM), modulates intestinal epithelial cell (IEC) growth and EBF. Controls were chow‐fed mice, T1 receptor‐3 (T1R3)‐knockout (KO) mice, and treatment with the metabotropic glutamate receptor (mGluR)‐5 antagonist MTEP. TPN significantly changed the amount of T1Rs, GLM receptors, and transporters, and GLM prevented these changes. GLM significantly prevented TPN‐associated intestinal atrophy (2.5‐fold increase in IEC proliferation) and was dependent on up‐regulation of the protein kinase pAkt, but independent of T1R3 and mGluR5 signaling. GLM led to a loss of EBF with TPN (60% increase in FITC‐dextran permeability, 40% decline in transepithelial resistance); via T1R3, it protected EBF, whereas mGluR5 was associated with EBF loss. GLM led to a decline in circulating glucagon‐like peptide 2 (GLP‐2) during TPN. The decline was regulated by T1R3 and mGluR5, suggesting a novel negative regulator pathway for IEC proliferation not previously described. Loss of luminal nutrients with TPN administration may widely affect intestinal taste sensing. GLM has previously unrecognized actions on IEC growth and EBF. Restoring luminal sensing via GLM could be a strategy for patients on TPN.—Xiao, W., Feng, Y., Holst, J. J., Hartmann, B., Yang, H., Teitelbaum, D. H. Glutamate prevents intestinal atrophy via luminal nutrient sensing in a mouse model of total parenteral nutrition. FASEB J. 28, 2073–2087 (2014). www.fasebj.orgPeer Reviewedhttps://deepblue.lib.umich.edu/bitstream/2027.42/154477/1/fsb2fj13238311.pdfhttps://deepblue.lib.umich.edu/bitstream/2027.42/154477/2/fsb2fj13238311-sup-0001.pd

Source placement error for permanent implant of the prostate

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/134896/1/mp8058.pd

Self-reported intake of high-fat and high-sugar diet is not associated with cognitive stability and flexibility in healthy men

Animal studies indicate that a high-fat/high-sugar diet (HFS) can change dopamine signal transmission in the brain, which could promote maladaptive behavior and decision-making. Such diet-induced changes may also explain observed alterations in the dopamine system in human obesity. Genetic variants that modulate dopamine transmission have been proposed to render some individuals more prone to potential effects of HFS. The objective of this study was to investigate the association of HFS with dopamine-dependent cognition in humans and how genetic variations might modulate this potential association. Using a questionnaire assessing the self -reported consumption of high-fat/high-sugar foods, we investigated the association with diet by recruiting healthy young men that fall into the lower or upper end of that questionnaire (low fat/sugar group: LFS, n = 45; high fat/sugar group: HFS, n = 41) and explored the interaction of fat and sugar consumption with COMT Va1158Met and Taq1A genotype. During functional magnetic resonance imaging (fMRI) scanning, male partici-pants performed a working memory (WM) task that probes distractor-resistance and updating of WM repre-sentations. Logistic and linear regression models revealed no significant difference in WM performance between the two diet groups, nor an interaction with COMT Va1t58Met or Tag1A genotype. Neural activation in task -related brain areas also did not differ between diet groups. Independent of diet group, higher BMI was associ-ated with lower overall accuracy on the WM task. This cross-sectional study does not provide evidence for diet -related differences in WM stability and flexibility in men, nor for a predisposition of COMT Va1158Met or Tag1A genotype to the hypothesized detrimental effects of an HFS diet. Previously reported associations of BMI with WM seem to be independent of HFS intake in our male study sample.Peer reviewe

Spin-labelled photo-cytotoxic diazido platinum(iv) anticancer complex

We report the synthesis and characterisation of the nitroxide spin-labelled photoactivatable Pt(IV) prodrug trans,trans,trans-[Pt(N3)2(OH)(OCOCH2CH2CONH-TEMPO)(Py)2] (Pt-TEMPO, where TEMPO = 2,2,6,6-tetramethylpiperidine 1-oxyl). Irradiation with blue visible light gave rise to Pt(II) and azidyl as well as nitroxyl radicals. Pt-TEMPO exhibited low toxicity in the dark, but on photoactivation was as active towards human ovarian cancer cells as the clinical photosensitizer chlorpromazine and much more active than the anticancer drug cisplatin under the conditions used

Toward a systematic 1/d expansion: Two particle properties

We present a procedure to calculate 1/d corrections to the two-particle properties around the infinite dimensional dynamical mean field limit. Our method is based on a modified version of the scheme of Ref. onlinecite{SchillerIngersent}}. To test our method we study the Hubbard model at half filling within the fluctuation exchange approximation (FLEX), a selfconsistent generalization of iterative perturbation theory. Apart from the inherent unstabilities of FLEX, our method is stable and results in causal solutions. We find that 1/d corrections to the local approximation are relatively small in the Hubbard model.Comment: 4 pages, 4 eps figures, REVTe