Projected Off-Policy Q-Learning (POP-QL) for Stabilizing Offline Reinforcement Learning

A key problem in off-policy Reinforcement Learning (RL) is the mismatch, or distribution shift, between the dataset and the distribution over states and actions visited by the learned policy. This problem is exacerbated in the fully offline setting. The main approach to correct this shift has been through importance sampling, which leads to high-variance gradients. Other approaches, such as conservatism or behavior-regularization, regularize the policy at the cost of performance. In this paper, we propose a new approach for stable off-policy Q-Learning. Our method, Projected Off-Policy Q-Learning (POP-QL), is a novel actor-critic algorithm that simultaneously reweights off-policy samples and constrains the policy to prevent divergence and reduce value-approximation error. In our experiments, POP-QL not only shows competitive performance on standard benchmarks, but also out-performs competing methods in tasks where the data-collection policy is significantly sub-optimal.Comment: 10 page

Prevalence and Risk Factors of Gastrointestinal Disorders in Patients with Rheumatoid Arthritis: Results from a Population-Based Survey in Olmsted County, Minnesota

Objectives. To compare the prevalence of gastrointestinal (GI) disorders in rheumatoid arthritis (RA) versus non-RA subjects and to describe determinants of GI disorders in RA. Methods. The bowel disease questionnaire was completed by RA and non-RA subjects. RA patients also completed the health assessment questionnaire (HAQ). Results. The study responders included 284 RA and 233 non-RA subjects. Abdominal pain/discomfort, postprandial fullness, nausea, and stool leakage were significantly more common in RA versus non-RA (odds ratios [OR] = 1.8; 1.9; 4.0; 8.2, resp.). The use of laxatives, proton pump inhibitors, NSAIDs, acetaminophen, and narcotics was more commonly reported in RA versus non-RA (OR = 2.0; 1.7; 3.0; 2.0; 1.9, resp.). Age < 60 and HAQ ≥ 1 were associated with dyspepsia, irritable bowel syndrome, gastroesophageal reflux disease, and GI symptom complex overlap in RA. Conclusion. Several upper and lower GI disorders were significantly more prevalent in RA versus non-RA subjects. Age <60 and physical function impairment (HAQ ≥ 1) were associated with GI disorders in RA

Gender-Based Differences Among Pharmacy Students Involved in Academically Dishonest Behavior

Objective. To determine whether differences based on gender exist among pharmacy students involved in cases of admitted cheating or other academic dishonesty and to assess perceptions of academic dishonesty. Methods. Two cohorts of second-year male and female pharmacy students from four Northern California pharmacy programs were invited to complete a 45-item cross-sectional survey. Descriptive statistics and Pearson’s chi-squared test were used for statistical analysis. Results. There were 330 surveys completed with a 59% response rate. No significant gender-based differences were found regarding admitted cheating in pharmacy school and in regards to participating in various forms of academically dishonest behavior. Female students were more likely than male students to report witnessing a classmate copying another student’s assignment. Male students were less likely than female students to perceive a student who distributed a stolen exam as a cheater. Conclusion. No gender-based differences were noted in cases of admitted cheating or with regards to taking part in various forms of academically dishonest behavior. However, female students report witnessing cheating more than male students, and male students may have a more lenient perception toward academically dishonest behavior than female students. The information gathered from this study may provide further insight to pharmacy programs and educators regarding academic dishonesty at their institution

What makes brands achieve iconic status?

We propose that brands do not achieve iconic status by chance. This article focuses on how brands manage iconic status effectively. Drawing on an exploratory study of iconic brands, we identify a brand's ability to inspire consumers and connect with them on a personal level as well as its visual identity and presence in consumers' mind as critical elements of brand status. Consumers' perceptions of a sample of brands were investigated through in-depth interviews, followed by the examination of these brands' activities through case-study analyses. The alignment between brand strategies and the relevant features highlighted by consumers was then assessed. A comprehensive framework for achieving iconicity is presented and discussed.Working Pape

Data on 824 fireballs observed by the digital cameras of the European Fireball Network in 2017-2018. I. Description of the network, data reduction procedures, and the catalog

A catalog of 824 fireballs (bright meteors), observed by a dedicated network of all-sky digital photographic cameras in central Europe in the years 2017-2018 is presented. The status of the European Fireball Network, established in 1963, is described. The cameras collect digital images of meteors brighter than an absolute magnitude of about -2 and radiometric light curves with a high temporal resolution of those brighter than a magnitude ~ -4. All meteoroids larger than 5 grams, corresponding to sizes of about 2 cm, are detected regardless of their entry velocity. High-velocity meteoroids are detected down to masses of about 0.1 gram. The largest observed meteoroid in the reported period 2017-2018 had a mass of about 100 kg and a size of about 40 cm. The methods of data analysis are explained and all catalog entries are described in detail. The provided data include the fireball date and time, atmospheric trajectory and velocity, the radiant in various coordinate systems, heliocentric orbital elements, maximum brightness, radiated energy, initial and terminal masses, maximum encountered dynamic pressure, physical classification, and possible shower membership. Basic information on the fireball spectrum is available for some bright fireballs (apparent magnitude < -7). A simple statistical evaluation of the whole sample is provided. The scientific analysis is presented in an accompanying paper.Comment: accepted in Astronomy and Astrophysic

Toroidal optical dipole traps for atomic Bose-Einstein condensates using Laguerre-Gaussian beams

We theoretically investigate the use of red-detuned Laguerre-Gaussian (LG) laser beams of varying azimuthal mode index for producing toroidal optical dipole traps in two-dimensional atomic Bose-Einstein condensates. Higher-order LG beams provide deeper potential wells and tighter confinement for a fixed toroid radius and laser power. Numerical simulations of the loading of the toroidal trap from a variety of initial conditions is also given.Comment: 12 pages, 5 figures, submitted to Phys. Rev.

Cold atom gas at very high densities in an optical surface microtrap

An optical microtrap is realized on a dielectric surface by crossing a tightly focused laser beam with an horizontal evanescent-wave atom mirror. The nondissipative trap is loaded with $\sim$$10^5$ cesium atoms through elastic collisions from a cold reservoir provided by a large-volume optical surface trap. With an observed 300-fold local increase of the atomic number density approaching $10^{14}{\rm cm}^{-3}$, unprecedented conditions of cold atoms close to a surface are realized

Aberration-free ultra-thin flat lenses and axicons at telecom wavelengths based on plasmonic metasurfaces

The concept of optical phase discontinuities is applied to the design and demonstration of aberration-free planar lenses and axicons, comprising a phased array of ultrathin subwavelength spaced optical antennas. The lenses and axicons consist of radial distributions of V-shaped nanoantennas that generate respectively spherical wavefronts and non-diffracting Bessel beams at telecom wavelengths. Simulations are also presented to show that our aberration-free designs are applicable to high numerical aperture lenses such as flat microscope objectives