Learning Contextual Bandits in a Non-stationary Environment

Multi-armed bandit algorithms have become a reference solution for handling the explore/exploit dilemma in recommender systems, and many other important real-world problems, such as display advertisement. However, such algorithms usually assume a stationary reward distribution, which hardly holds in practice as users' preferences are dynamic. This inevitably costs a recommender system consistent suboptimal performance. In this paper, we consider the situation where the underlying distribution of reward remains unchanged over (possibly short) epochs and shifts at unknown time instants. In accordance, we propose a contextual bandit algorithm that detects possible changes of environment based on its reward estimation confidence and updates its arm selection strategy respectively. Rigorous upper regret bound analysis of the proposed algorithm demonstrates its learning effectiveness in such a non-trivial environment. Extensive empirical evaluations on both synthetic and real-world datasets for recommendation confirm its practical utility in a changing environment.Comment: 10 pages, 13 figures, To appear on ACM Special Interest Group on Information Retrieval (SIGIR) 201

Pulsars as Calibration Tools and X-Ray Observations of Spider Pulsars

We present the polarization pulse profiles for 29 pulsars observed with the Arecibo Observatory by the North American Nanohertz Observatory for Gravitational Waves (NANOGrav) timing project at 2.1 GHz, 1.4 GHz, and 430 MHz. These profiles represent the most sensitive polarimetric millisecond pulsar profiles to date, revealing the existence of microcomponents (that is, pulse components with peak intensities much lower than the total pulse peak intensity). Although microcomponents have been detected in some pulsars previously, we are able to detect new microcomponents for PSRs B1937+21, J1713+0747, and J2234+0944. We also present rotation measures for 28 of these pulsars, determined independently at different observation frequencies and epochs, and find the Galactic magnetic fields derived from these rotation measures to be consistent with current models. These polarization profiles were made using measurement equation template matching, which allows us to generate the polarimetric response of the Arecibo Observatory on an epoch-by-epoch basis. We use this method to describe its time variability, and find that the polarimetric responses of the Arecibo Observatory\u27s 1.4 and 2.1 GHz receivers varies significantly with time.;We then describe the first X-ray observations of five short orbital period (PB \u3c 1 day), gamma-ray emitting, binary millisecond pulsars. Four of these---PSRs J0023+0923, J1124--3653, J1810+1744, and J2256--1024---are black-widow pulsars, with degenerate companions of mass 0.1 solar mass, three of which exhibit radio eclipses. The fifth source, PSR J2215+5135, is an eclipsing redback with a near Roche-lobe filling 0.2 solar mass non-degenerate companion. Data were taken using the Chandra X-Ray Observatory and covered a full binary orbit for each pulsar. Two pulsars, PSRs J2215+5135 and J2256--1024, show significant orbital variability while PSR J1124--3653 shows marginal orbital variability. The lightcurves for these three pulsars have X-ray flux minima coinciding with the phases of the radio eclipses. This phenomenon is consistent with an intrabinary shock emission interpretation for the X-rays. The other two pulsars, PSRs J0023+0923 and J1810+1744, are fainter and do not demonstrate variability at a level we can detect in these data. All five spectra are fit with three separate models: a power-law model, a blackbody model, and a combined model with both power-law and blackbody components. The preferred spectral fits yield power-law indices that range from 1.3 to 3.2 and blackbody temperatures in the hundreds of eV. The spectrum for PSR J2215+5135 shows a significant hard X-ray component, with a large number of counts above 2 keV, which is additional evidence for the presence of intrabinary shock emission. This is similar to what has been detected in the low-mass X-ray binary to MSP transition object PSR J1023+0038.;We also describe X-Ray observations of three redback pulsars taken with the XMM--Newton X-Ray telescope, and cover at least one orbit for each source. We had previously analyzed data for one of these sources, PSR~J2215+5135, taken with the Chandra X-Ray Observatory . These new observations also show orbital variability in PSR~J2215+5135\u27s X-Ray lightcurve, including an X-Ray minimum near superior conjunction, and the increased sensitivity allows us to see two clear features away from superior conjunction. For the other two sources, PSRs J1622--0315 and J1908+2105, we do not detect enough counts to constrain the X-Ray orbital variability. The spectra for each of these sources showed significant hard X-Ray emission, and were therefore not well described by thermal models. We report power-law indices from these fits in the range of 1.28 to 2.0. These spectral properties are consistent with intrabinary shock emission

The distribution of interstellar dust in CALIFA edge-on galaxies via oligochromatic radiative transfer fitting

We investigate the amount and spatial distribution of interstellar dust in edge-on spiral galaxies, using detailed radiative transfer modeling of a homogeneous sample of 12 galaxies selected from the CALIFA survey. Our automated fitting routine, FitSKIRT, was first validated against artificial data. This is done by simultaneously reproducing the SDSS $g$-, $r$-, $i$- and $z$-band observations of a toy model in order to combine the information present in the different bands. We show that this combined, oligochromatic fitting, has clear advantages over standard monochromatic fitting especially regarding constraints on the dust properties. We model all galaxies in our sample using a three-component model, consisting of a double exponential disc to describe the stellar and dust discs and using a S\'ersic profile to describe the central bulge. The full model contains 19 free parameters, and we are able to constrain all these parameters to a satisfactory level of accuracy without human intervention or strong boundary conditions. Apart from two galaxies, the entire sample can be accurately reproduced by our model. We find that the dust disc is about 75% more extended but only half as high as the stellar disc. The average face-on optical depth in the V-band is $0.76$ and the spread of $0.60$ within our sample is quite substantial, which indicates that some spiral galaxies are relatively opaque even when seen face-on.Comment: 18 pages, 6 figures, 4 tables, Accepted for publication in MNRA