Actor-Critic Policy Learning in Cooperative Planning

In this paper, we introduce a method for learning and adapting cooperative control strategies in real-time stochastic domains. Our framework is an instance of the intelligent cooperative control architecture (iCCA)[superscript 1]. The agent starts by following the "safe" plan calculated by the planning module and incrementally adapting its policy to maximize the cumulative rewards. Actor-critic and consensus-based bundle algorithm (CBBA) were employed as the building blocks of the iCCA framework. We demonstrate the performance of our approach by simulating limited fuel unmanned aerial vehicles aiming for stochastic targets. In one experiment where the optimal solution can be calculated, the integrated framework boosted the optimality of the solution by an average of %10, when compared to running each of the modules individually, while keeping the computational load within the requirements for real-time implementation.Boeing Scientific Research LaboratoriesUnited States. Air Force Office of Scientific Research (Grant FA9550-08-1-0086

Basis Expansion in Natural Actor Critic Methods

International audienceIn reinforcement learning, the aim of the agent is to find a policy that maximizes its expected return. Policy gradient methods try to accomplish this goal by directly approximating the policy using a parametric function approximator; the expected return of the current policy is estimated and its parameters are updated by steepest ascent in the direction of the gradient of the expected return with respect to the policy parameters. In general, the policy is defined in terms of a set of basis functions that capture important features of the problem. Since the quality of the resulting policies directly depend on the set of basis func- tions, and defining them gets harder as the complexity of the problem increases, it is important to be able to find them automatically. In this paper, we propose a new approach which uses cascade-correlation learn- ing architecture for automatically constructing a set of basis functions within the context of Natural Actor-Critic (NAC) algorithms. Such basis functions allow more complex policies be represented, and consequently improve the performance of the resulting policies. We also present the effectiveness of the method empirically

Phase separating binary fluids under oscillatory shear

We apply lattice Boltzmann methods to study the segregation of binary fluid mixtures under oscillatory shear flow in two dimensions. The algorithm allows to simulate systems whose dynamics is described by the Navier-Stokes and the convection-diffusion equations. The interplay between several time scales produces a rich and complex phenomenology. We investigate the effects of different oscillation frequencies and viscosities on the morphology of the phase separating domains. We find that at high frequencies the evolution is almost isotropic with growth exponents 2/3 and 1/3 in the inertial (low viscosity) and diffusive (high viscosity) regimes, respectively. When the period of the applied shear flow becomes of the same order of the relaxation time $T_R$ of the shear velocity profile, anisotropic effects are clearly observable. In correspondence with non-linear patterns for the velocity profiles, we find configurations where lamellar order close to the walls coexists with isotropic domains in the middle of the system. For particular values of frequency and viscosity it can also happen that the convective effects induced by the oscillations cause an interruption or a slowing of the segregation process, as found in some experiments. Finally, at very low frequencies, the morphology of domains is characterized by lamellar order everywhere in the system resembling what happens in the case with steady shear.Comment: 1 table and 12 figures in .gif forma

Glauber Dynamics for the mean-field Potts Model

We study Glauber dynamics for the mean-field (Curie-Weiss) Potts model with $q\geq 3$ states and show that it undergoes a critical slowdown at an inverse-temperature $\beta_s(q)$ strictly lower than the critical $\beta_c(q)$ for uniqueness of the thermodynamic limit. The dynamical critical $\beta_s(q)$ is the spinodal point marking the onset of metastability. We prove that when $\beta<\beta_s(q)$ the mixing time is asymptotically $C(\beta, q) n \log n$ and the dynamics exhibits the cutoff phenomena, a sharp transition in mixing, with a window of order $n$. At $\beta=\beta_s(q)$ the dynamics no longer exhibits cutoff and its mixing obeys a power-law of order $n^{4/3}$. For $\beta>\beta_s(q)$ the mixing time is exponentially large in $n$. Furthermore, as $\beta \uparrow \beta_s$ with $n$, the mixing time interpolates smoothly from subcritical to critical behavior, with the latter reached at a scaling window of $O(n^{-2/3})$ around $\beta_s$. These results form the first complete analysis of mixing around the critical dynamical temperature --- including the critical power law --- for a model with a first order phase transition.Comment: 45 pages, 5 figure

Search for a W' boson decaying to a bottom quark and a top quark in pp collisions at sqrt(s) = 7 TeV

Results are presented from a search for a W' boson using a dataset corresponding to 5.0 inverse femtobarns of integrated luminosity collected during 2011 by the CMS experiment at the LHC in pp collisions at sqrt(s)=7 TeV. The W' boson is modeled as a heavy W boson, but different scenarios for the couplings to fermions are considered, involving both left-handed and right-handed chiral projections of the fermions, as well as an arbitrary mixture of the two. The search is performed in the decay channel W' to t b, leading to a final state signature with a single lepton (e, mu), missing transverse energy, and jets, at least one of which is tagged as a b-jet. A W' boson that couples to fermions with the same coupling constant as the W, but to the right-handed rather than left-handed chiral projections, is excluded for masses below 1.85 TeV at the 95% confidence level. For the first time using LHC data, constraints on the W' gauge coupling for a set of left- and right-handed coupling combinations have been placed. These results represent a significant improvement over previously published limits.Comment: Submitted to Physics Letters B. Replaced with version publishe

Search for the standard model Higgs boson decaying into two photons in pp collisions at sqrt(s)=7 TeV

A search for a Higgs boson decaying into two photons is described. The analysis is performed using a dataset recorded by the CMS experiment at the LHC from pp collisions at a centre-of-mass energy of 7 TeV, which corresponds to an integrated luminosity of 4.8 inverse femtobarns. Limits are set on the cross section of the standard model Higgs boson decaying to two photons. The expected exclusion limit at 95% confidence level is between 1.4 and 2.4 times the standard model cross section in the mass range between 110 and 150 GeV. The analysis of the data excludes, at 95% confidence level, the standard model Higgs boson decaying into two photons in the mass range 128 to 132 GeV. The largest excess of events above the expected standard model background is observed for a Higgs boson mass hypothesis of 124 GeV with a local significance of 3.1 sigma. The global significance of observing an excess with a local significance greater than 3.1 sigma anywhere in the search range 110-150 GeV is estimated to be 1.8 sigma. More data are required to ascertain the origin of this excess.Comment: Submitted to Physics Letters

Measurement of the Lambda(b) cross section and the anti-Lambda(b) to Lambda(b) ratio with Lambda(b) to J/Psi Lambda decays in pp collisions at sqrt(s) = 7 TeV

The Lambda(b) differential production cross section and the cross section ratio anti-Lambda(b)/Lambda(b) are measured as functions of transverse momentum pt(Lambda(b)) and rapidity abs(y(Lambda(b))) in pp collisions at sqrt(s) = 7 TeV using data collected by the CMS experiment at the LHC. The measurements are based on Lambda(b) decays reconstructed in the exclusive final state J/Psi Lambda, with the subsequent decays J/Psi to an opposite-sign muon pair and Lambda to proton pion, using a data sample corresponding to an integrated luminosity of 1.9 inverse femtobarns. The product of the cross section times the branching ratio for Lambda(b) to J/Psi Lambda versus pt(Lambda(b)) falls faster than that of b mesons. The measured value of the cross section times the branching ratio for pt(Lambda(b)) > 10 GeV and abs(y(Lambda(b))) < 2.0 is 1.06 +/- 0.06 +/- 0.12 nb, and the integrated cross section ratio for anti-Lambda(b)/Lambda(b) is 1.02 +/- 0.07 +/- 0.09, where the uncertainties are statistical and systematic, respectively.Comment: Submitted to Physics Letters

Search for new physics in events with opposite-sign leptons, jets, and missing transverse energy in pp collisions at sqrt(s) = 7 TeV

A search is presented for physics beyond the standard model (BSM) in final states with a pair of opposite-sign isolated leptons accompanied by jets and missing transverse energy. The search uses LHC data recorded at a center-of-mass energy sqrt(s) = 7 TeV with the CMS detector, corresponding to an integrated luminosity of approximately 5 inverse femtobarns. Two complementary search strategies are employed. The first probes models with a specific dilepton production mechanism that leads to a characteristic kinematic edge in the dilepton mass distribution. The second strategy probes models of dilepton production with heavy, colored objects that decay to final states including invisible particles, leading to very large hadronic activity and missing transverse energy. No evidence for an event yield in excess of the standard model expectations is found. Upper limits on the BSM contributions to the signal regions are deduced from the results, which are used to exclude a region of the parameter space of the constrained minimal supersymmetric extension of the standard model. Additional information related to detector efficiencies and response is provided to allow testing specific models of BSM physics not considered in this paper.Comment: Replaced with published version. Added journal reference and DO

Biomarkers of exposure to new and emerging tobacco delivery products

Accurate and reliable measurements of exposure to tobacco products are essential for identifying and confirming patterns of tobacco product use and for assessing their potential biological effects in both human populations and experimental systems. Due to the introduction of new tobaccoderived products and the development of novel ways to modify and use conventional tobacco products, precise and specific assessments of exposure to tobacco are now more important than ever. Biomarkers that were developed and validated to measure exposure to cigarettes are being evaluated to assess their use for measuring exposure to these new products. Here, we review current methods for measuring exposure to new and emerging tobacco products, such as electronic cigarettes, little cigars, water pipes, and cigarillos. Rigorously validated biomarkers specific to these new products have not yet been identified. Here, we discuss the strengths and limitations of current approaches, including whether they provide reliable exposure estimates for new and emerging products. We provide specific guidance for choosing practical and economical biomarkers for different study designs and experimental conditions. Our goal is to help both new and experienced investigators measure exposure to tobacco products accurately and avoid common experimental errors. With the identification of the capacity gaps in biomarker research on new and emerging tobacco products, we hope to provide researchers, policymakers, and funding agencies with a clear action plan for conducting and promoting research on the patterns of use and health effects of these products

Collisional and Radiative Processes in Optically Thin Plasmas

Most of our knowledge of the physical processes in distant plasmas is obtained through measurement of the radiation they produce. Here we provide an overview of the main collisional and radiative processes and examples of diagnostics relevant to the microphysical processes in the plasma. Many analyses assume a time-steady plasma with ion populations in equilibrium with the local temperature and Maxwellian distributions of particle velocities, but these assumptions are easily violated in many cases. We consider these departures from equilibrium and possible diagnostics in detail