Decision-Theoretic Planning with non-Markovian Rewards

A decision process in which rewards depend on history rather than merely on the current state is called a decision process with non-Markovian rewards (NMRDP). In decision-theoretic planning, where many desirable behaviours are more naturally expressed as properties of execution sequences rather than as properties of states, NMRDPs form a more natural model than the commonly adopted fully Markovian decision process (MDP) model. While the more tractable solution methods developed for MDPs do not directly apply in the presence of non-Markovian rewards, a number of solution methods for NMRDPs have been proposed in the literature. These all exploit a compact specification of the non-Markovian reward function in temporal logic, to automatically translate the NMRDP into an equivalent MDP which is solved using efficient MDP solution methods. This paper presents NMRDPP (Non-Markovian Reward Decision Process Planner), a software platform for the development and experimentation of methods for decision-theoretic planning with non-Markovian rewards. The current version of NMRDPP implements, under a single interface, a family of methods based on existing as well as new approaches which we describe in detail. These include dynamic programming, heuristic search, and structured methods. Using NMRDPP, we compare the methods and identify certain problem features that affect their performance. NMRDPPs treatment of non-Markovian rewards is inspired by the treatment of domain-specific search control knowledge in the TLPlan planner, which it incorporates as a special case. In the First International Probabilistic Planning Competition, NMRDPP was able to compete and perform well in both the domain-independent and hand-coded tracks, using search control knowledge in the latter

Atmospheric temperature responses to solar irradiance and geomagnetic activity

The relative effects of solar irradiance and geomagnetic activity on the atmospheric temperature anomalies (Ta) are examined from the monthly to interdecadal timescales. Geomagnetic Ap (Ap) signals are found primarily in the stratosphere, while the solar F10.7-cm radio flux (Fs) signals are found in both the stratosphere and troposphere. In the troposphere, 0.1–0.4 K increases in Ta are associated with Fs. Enhanced Fs signals are found when the stratospheric quasi-biennial oscillation (QBO) is westerly. In the extrapolar region of the stratosphere, 0.1–0.6 and 0.1–0.7 K increases in Ta are associated with solar irradiance and with geomagnetic activity, respectively. In this region, Fs signals are strengthened when either the QBO is easterly, or geomagnetic activity is high, while Ap signals are strengthened when either the QBO is westerly, or solar irradiance is high. High solar irradiance and geomagnetic activity tend to enhance each other's signatures either making the signals stronger and symmetric about the equator or extending the signals to broader areas, or both. Positive Ap signals dominate the middle Arctic stratosphere and are two to five times larger than those of Fs. When solar irradiance is low, the signature of Ap in Ta is asymmetric about the equator, with positive signals in the Arctic stratosphere and negative signals at midlatitudes of the NH stratosphere. Weaker stratospheric QBO signals are associated with high Ap and Fs, suggesting possible disturbances on the QBO. The signals of Ap and Fs are distinct from the positive temperature anomalies resulting from volcanic eruptions

A finite model of two-dimensional ideal hydrodynamics

A finite-dimensional su($N$) Lie algebra equation is discussed that in the infinite $N$ limit (giving the area preserving diffeomorphism group) tends to the two-dimensional, inviscid vorticity equation on the torus. The equation is numerically integrated, for various values of $N$, and the time evolution of an (interpolated) stream function is compared with that obtained from a simple mode truncation of the continuum equation. The time averaged vorticity moments and correlation functions are compared with canonical ensemble averages.Comment: (25 p., 7 figures, not included. MUTP/92/1

Continuous Interaction with a Virtual Human

Attentive Speaking and Active Listening require that a Virtual Human be capable of simultaneous perception/interpretation and production of communicative behavior. A Virtual Human should be able to signal its attitude and attention while it is listening to its interaction partner, and be able to attend to its interaction partner while it is speaking – and modify its communicative behavior on-the-fly based on what it perceives from its partner. This report presents the results of a four week summer project that was part of eNTERFACE’10. The project resulted in progress on several aspects of continuous interaction such as scheduling and interrupting multimodal behavior, automatic classification of listener responses, generation of response eliciting behavior, and models for appropriate reactions to listener responses. A pilot user study was conducted with ten participants. In addition, the project yielded a number of deliverables that are released for public access

Radiation hardness qualification of PbWO4 scintillation crystals for the CMS Electromagnetic Calorimeter

This is the Pre-print version of the Article. The official published version can be accessed from the link below - Copyright @ 2010 IOPEnsuring the radiation hardness of PbWO4 crystals was one of the main priorities during the construction of the electromagnetic calorimeter of the CMS experiment at CERN. The production on an industrial scale of radiation hard crystals and their certification over a period of several years represented a difficult challenge both for CMS and for the crystal suppliers. The present article reviews the related scientific and technological problems encountered

The CMS Electromagnetic Calorimeter Data Acquisition System at the 2006 Test Beam

The Electromagnetic Calorimeter of the CMS experiment at the CERN LHC is an homogeneous calorimeter made of about 80000 Lead Tungstate crystals. From June to November 2006, eleven barrel Supermodules (1700 crystals each) were exposed to beam at CERN SPS, both in stand-alone and in association with portions of the Hadron Calorimeter. We present the description of the system used to configure and readout the calorimeter during this period. The full set of final readout electronics boards was employed, together with the pre-series version of the data acquisition software. During this testbeam, the hardware and software concepts for the final system were validated and the successfull operation of all the ten supermodules was ensured

Measurement of the quasi-elastic axial vector mass in neutrino-oxygen interactions

The weak nucleon axial-vector form factor for quasi-elastic interactions is determined using neutrino interaction data from the K2K Scintillating Fiber detector in the neutrino beam at KEK. More than 12,000 events are analyzed, of which half are charged-current quasi-elastic interactions nu-mu n to mu- p occurring primarily in oxygen nuclei. We use a relativistic Fermi gas model for oxygen and assume the form factor is approximately a dipole with one parameter, the axial vector mass M_A, and fit to the shape of the distribution of the square of the momentum transfer from the nucleon to the nucleus. Our best fit result for M_A = 1.20 \pm 0.12 GeV. Furthermore, this analysis includes updated vector form factors from recent electron scattering experiments and a discussion of the effects of the nucleon momentum on the shape of the fitted distributions.Comment: 14 pages, 10 figures, 6 table

A Study of Time-Dependent CP-Violating Asymmetries and Flavor Oscillations in Neutral B Decays at the Upsilon(4S)

We present a measurement of time-dependent CP-violating asymmetries in neutral B meson decays collected with the BABAR detector at the PEP-II asymmetric-energy B Factory at the Stanford Linear Accelerator Center. The data sample consists of 29.7 ${\rm fb}^{-1}$ recorded at the $\Upsilon(4S)$ resonance and 3.9 ${\rm fb}^{-1}$ off-resonance. One of the neutral B mesons, which are produced in pairs at the $\Upsilon(4S)$, is fully reconstructed in the CP decay modes $J/\psi K^0_S$, $\psi(2S) K^0_S$, $\chi_{c1} K^0_S$, $J/\psi K^{*0}$ ($K^{*0}\to K^0_S\pi^0$) and $J/\psi K^0_L$, or in flavor-eigenstate modes involving $D^{(*)}\pi/\rho/a_1$ and $J/\psi K^{*0}$ ($K^{*0}\to K^+\pi^-$). The flavor of the other neutral B meson is tagged at the time of its decay, mainly with the charge of identified leptons and kaons. The proper time elapsed between the decays is determined by measuring the distance between the decay vertices. A maximum-likelihood fit to this flavor eigenstate sample finds $\Delta m_d = 0.516\pm 0.016 {\rm (stat)} \pm 0.010 {\rm (syst)} {\rm ps}^{-1}$. The value of the asymmetry amplitude $\sin2\beta$ is determined from a simultaneous maximum-likelihood fit to the time-difference distribution of the flavor-eigenstate sample and about 642 tagged $B^0$ decays in the CP-eigenstate modes. We find $\sin2\beta=0.59\pm 0.14 {\rm (stat)} \pm 0.05 {\rm (syst)}$, demonstrating that CP violation exists in the neutral B meson system. (abridged)Comment: 58 pages, 35 figures, submitted to Physical Review

Measurement of the Branching Fraction for B- --> D0 K*-

We present a measurement of the branching fraction for the decay B- --> D0 K*- using a sample of approximately 86 million BBbar pairs collected by the BaBar detector from e+e- collisions near the Y(4S) resonance. The D0 is detected through its decays to K- pi+, K- pi+ pi0 and K- pi+ pi- pi+, and the K*- through its decay to K0S pi-. We measure the branching fraction to be B.F.(B- --> D0 K*-)= (6.3 +/- 0.7(stat.) +/- 0.5(syst.)) x 10^{-4}.Comment: 7 pages, 1 postscript figure, submitted to Phys. Rev. D (Rapid Communications

Evidence for the Rare Decay B -> K*ll and Measurement of the B -> Kll Branching Fraction

We present evidence for the flavor-changing neutral current decay $B\to K^*\ell^+\ell^-$ and a measurement of the branching fraction for the related process $B\to K\ell^+\ell^-$, where $\ell^+\ell^-$ is either an $e^+e^-$ or $\mu^+\mu^-$ pair. These decays are highly suppressed in the Standard Model, and they are sensitive to contributions from new particles in the intermediate state. The data sample comprises $123\times 10^6$ $\Upsilon(4S)\to B\bar{B}$ decays collected with the Babar detector at the PEP-II $e^+e^-$ storage ring. Averaging over $K^{(*)}$ isospin and lepton flavor, we obtain the branching fractions ${\mathcal B}(B\to K\ell^+\ell^-)=(0.65^{+0.14}_{-0.13}\pm 0.04)\times 10^{-6}$ and ${\mathcal B}(B\to K^*\ell^+\ell^-)=(0.88^{+0.33}_{-0.29}\pm 0.10)\times 10^{-6}$, where the uncertainties are statistical and systematic, respectively. The significance of the $B\to K\ell^+\ell^-$ signal is over $8\sigma$, while for $B\to K^*\ell^+\ell^-$ it is $3.3\sigma$.Comment: 7 pages, 2 postscript figues, submitted to Phys. Rev. Let