Solving Factored MDPs with Hybrid State and Action Variables

Efficient representations and solutions for large decision problems with continuous and discrete variables are among the most important challenges faced by the designers of automated decision support systems. In this paper, we describe a novel hybrid factored Markov decision process (MDP) model that allows for a compact representation of these problems, and a new hybrid approximate linear programming (HALP) framework that permits their efficient solutions. The central idea of HALP is to approximate the optimal value function by a linear combination of basis functions and optimize its weights by linear programming. We analyze both theoretical and computational aspects of this approach, and demonstrate its scale-up potential on several hybrid optimization problems

A graphene-based glycan biosensor for electrochemical label-free detection of a tumor-associated antibody

The study describes development of a glycan biosensor for detection of a tumor-associated antibody. The glycan biosensor is built on an electrochemically activated/oxidized graphene screen-printed electrode (GSPE). Oxygen functionalities were subsequently applied for covalent immobilization of human serum albumin (HSA) as a natural nanoscaffold for covalent immobilization of Thomsen-nouvelle (Tn) antigen (GalNAc-O-Ser/Thr) to be fully available for affinity interaction with its analyte—a tumor-associated antibody. The step by step building process of glycan biosensor development was comprehensively characterized using a battery of techniques (scanning electron microscopy, atomic force microscopy, contact angle measurements, secondary ion mass spectrometry, surface plasmon resonance, Raman and energy-dispersive X-ray spectroscopy). Results suggest that electrochemical oxidation of graphene SPE preferentially oxidizes only the surface of graphene flakes within the graphene SPE. Optimization studies revealed the following optimal parameters: activation potential of +1.5 V vs. Ag/AgCl/3 M KCl, activation time of 60 s and concentration of HSA of 0.1 g L−1. Finally, the glycan biosensor was built up able to selectively and sensitively detect its analyte down to low aM concentration. The binding preference of the glycan biosensor was in an agreement with independent surface plasmon resonance analysis.The financial support received from the Slovak Scientific Grant Agency VEGA 2/0137/18 and 2/0090/16 from the Slovak Research and Development Agency APVV 17-0300 is acknowledged. This publication is the result of the project implementation: Centre for materials, layers and systems for applications and chemical processes under extreme conditions—Stage I, ITMS no.: 26240120007, supported by the ERDF. This publication was supported by Qatar University Collaborative Grant QUCG-CAM-19/20-2. The findings achieved herein are solely the responsibility of the authors.Scopu

BubbleRank: Safe Online Learning to Re-Rank via Implicit Click Feedback

In this paper, we study the problem of safe online learning to re-rank, where user feedback is used to improve the quality of displayed lists. Learning to rank has traditionally been studied in two settings. In the offline setting, rankers are typically learned from relevance labels created by judges. This approach has generally become standard in industrial applications of ranking, such as search. However, this approach lacks exploration and thus is limited by the information content of the offline training data. In the online setting, an algorithm can experiment with lists and learn from feedback on them in a sequential fashion. Bandit algorithms are well-suited for this setting but they tend to learn user preferences from scratch, which results in a high initial cost of exploration. This poses an additional challenge of safe exploration in ranked lists. We propose BubbleRank, a bandit algorithm for safe re-ranking that combines the strengths of both the offline and online settings. The algorithm starts with an initial base list and improves it online by gradually exchanging higher-ranked less attractive items for lower-ranked more attractive items. We prove an upper bound on the n-step regret of BubbleRank that degrades gracefully with the quality of the initial base list. Our theoretical findings are supported by extensive experiments on a large-scale real-world click dataset

Transverse-momentum-dependent Multiplicities of Charged Hadrons in Muon-Deuteron Deep Inelastic Scattering

A semi-inclusive measurement of charged hadron multiplicities in deep inelastic muon scattering off an isoscalar target was performed using data collected by the COMPASS Collaboration at CERN. The following kinematic domain is covered by the data: photon virtuality $Q^{2}>1$ (GeV/$c$)$^2$, invariant mass of the hadronic system $W > 5$ GeV/$c^2$, Bjorken scaling variable in the range $0.003 < x < 0.4$, fraction of the virtual photon energy carried by the hadron in the range $0.2 < z < 0.8$, square of the hadron transverse momentum with respect to the virtual photon direction in the range 0.02 (GeV/$c)^2 < P_{\rm{hT}}^{2} < 3$ (GeV/$c$)$^2$. The multiplicities are presented as a function of $P_{\rm{hT}}^{2}$ in three-dimensional bins of $x$, $Q^2$, $z$ and compared to previous semi-inclusive measurements. We explore the small-$P_{\rm{hT}}^{2}$ region, i.e. $P_{\rm{hT}}^{2} < 1$ (GeV/$c$)$^2$, where hadron transverse momenta are expected to arise from non-perturbative effects, and also the domain of larger $P_{\rm{hT}}^{2}$, where contributions from higher-order perturbative QCD are expected to dominate. The multiplicities are fitted using a single-exponential function at small $P_{\rm{hT}}^{2}$ to study the dependence of the average transverse momentum $\langle P_{\rm{hT}}^{2}\rangle$ on $x$, $Q^2$ and $z$. The power-law behaviour of the multiplicities at large $P_{\rm{hT}}^{2}$ is investigated using various functional forms. The fits describe the data reasonably well over the full measured range.Comment: 28 pages, 20 figure

Constraints on the χ_(c1) versus χ_(c2) polarizations in proton-proton collisions at √s = 8 TeV

The polarizations of promptly produced χ_(c1) and χ_(c2) mesons are studied using data collected by the CMS experiment at the LHC, in proton-proton collisions at √s=8  TeV. The χ_c states are reconstructed via their radiative decays χ_c → J/ψγ, with the photons being measured through conversions to e⁺e⁻, which allows the two states to be well resolved. The polarizations are measured in the helicity frame, through the analysis of the χ_(c2) to χ_(c1) yield ratio as a function of the polar or azimuthal angle of the positive muon emitted in the J/ψ → μ⁺μ⁻ decay, in three bins of J/ψ transverse momentum. While no differences are seen between the two states in terms of azimuthal decay angle distributions, they are observed to have significantly different polar anisotropies. The measurement favors a scenario where at least one of the two states is strongly polarized along the helicity quantization axis, in agreement with nonrelativistic quantum chromodynamics predictions. This is the first measurement of significantly polarized quarkonia produced at high transverse momentum