Utility-Based Reinforcement Learning: Unifying Single-objective and Multi-objective Reinforcement Learning

Research in multi-objective reinforcement learning (MORL) has introduced the utility-based paradigm, which makes use of both environmental rewards and a function that defines the utility derived by the user from those rewards. In this paper we extend this paradigm to the context of single-objective reinforcement learning (RL), and outline multiple potential benefits including the ability to perform multi-policy learning across tasks relating to uncertain objectives, risk-aware RL, discounting, and safe RL. We also examine the algorithmic implications of adopting a utility-based approach.Comment: Accepted for the Blue Sky Track at AAMAS'2

On the asymptotic behaviour of solutions to the fractional porous medium equation with variable density

We are concerned with the long time behaviour of solutions to the fractional porous medium equation with a variable spatial density. We prove that if the density decays slowly at infinity, then the solution approaches the Barenblatt-type solution of a proper singular fractional problem. If, on the contrary, the density decays rapidly at infinity, we show that the minimal solution multiplied by a suitable power of the time variable converges to the minimal solution of a certain fractional sublinear elliptic equation.Comment: To appear in DCDS-

A Practical Guide to Multi-Objective Reinforcement Learning and Planning

Real-world decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems

Perturbed nonlocal fourth order equations of Kirchhoff type with Navier boundary conditions

Abstract We investigate the existence of multiple solutions for perturbed nonlocal fourth-order equations of Kirchhoff type under Navier boundary conditions. We give some new criteria for guaranteeing that the perturbed fourth-order equations of Kirchhoff type have at least three weak solutions by using a variational method and some critical point theorems due to Ricceri. We extend and improve some recent results. Finally, by presenting two examples, we ensure the applicability of our results

Noble gas and carbon isotope systematics at the seemingly inactive Ciomadul volcano (Eastern‐Central Europe, Romania): evidence for volcanic degassing

Ciomadul is the youngest volcano in the Carpathian-Pannonian Region, Eastern-Central Europe, which last erupted 30 ka. This volcano is considered to be inactive, however, combined evidence from petrologic and magnetotelluric data, as well as seismic tomography studies suggest the existence of a subvolcanic crystal mush with variable melt content. The volcanic area is characterized by high CO2 gas output rate, with a minimum of 8.7 × 103 t yr-1. We investigated 31 gas emissions at Ciomadul to constrain the origin of the volatiles. The δ13C-CO2 and 3He/4He compositions suggest the outgassing of a significant component of mantle-derived fluids. The He isotope signature in the outgassing fluids (up to 3.10 Ra) is lower than the values in the peridotite xenoliths of the nearby alkaline basalt volcanic field (R/Ra 5.95Ra±0.01) which are representative of a continental lithospheric mantle and significantly lower than MORB values. Considering the chemical characteristics of the Ciomadul dacite, including trace element and Sr- Nd and O isotope compositions, an upper crustal contamination is less probable, whereas the primary magmas could have been derived from an enriched mantle source. The low He isotopic ratios could indicate a strongly metasomatized mantle lithosphere. This could be due to infiltration of subduction-related fluids and postmetasomatic ingrowth of radiogenic He. The metasomatic fluids are inferred to have contained subducted carbonate material resulting in a heavier carbon isotope composition (13C is in the range of -1.4 to -4.6 ‰) and an increase of CO2/3He ratio. Our study shows the magmatic contribution to the emitted gases

Stable equivalence of the weak closures of free groups convolution algebras

We prove in this paper that the von Neumann algebras associated to the free non-commutative groups are stably isomorphic, i.e. that they are isomorphic when tensorized by the algebra of all linear bounded operators on a separable, infinite dimensional Hilbert space. This gives positive evidence for an old question, due to R.V. Kadison (see also S. Sakai's book on W*-algebras), whether the von Neumann algebras associated to free groups are isomorphic or not

Finite generation properties for fuchsian group von Neumann algebras tensor

We prove that the algebra $\mathcal{A}=\mathcal{L}(F_{N})\otimes B(H)$, $F_{N}$ a free group with finitely many generators, contains a subnormal operator $J$ such that the linear span of the set $\{(J^{*})^{n}J^{m}\vert n,m=0,1,2,...\}$ is weakly dense in $\mathcal{A}$. This is the analogue for the $II_{\infty }$ factor $\mathcal{L}(F_{N})\otimes B(H)$, $N$ finite, of a well known fact about the unilateral shift $S$ on a Hilbert space $K$: the linear span of all the monomials $(S^{*})^{n} S^{m}$ is weakly dense in $B(K)$. We also show that for a suitable space $H^{2}$ of square summable analytic functions, if $P$ is the projection from the Hilbert space $L^{2}$ of all square summable functions onto $H^{2}$ and $M_{\overline{j}}$ is the unbounded operator of multiplication by $\overline{j}$ on $L^{2}$, then the (unbounded) operator $PM_{\overline{j}}(I-P)$ is nonzero (with nonzero domain)