New prioritized value iteration for Markov decision processes

The problem of solving large Markov decision processes accurately and quickly is challenging. Since the computational effort incurred is considerable, current research focuses on finding superior acceleration techniques. For instance, the convergence properties of current solution methods depend, to a great extent, on the order of backup operations. On one hand, algorithms such as topological sorting are able to find good orderings but their overhead is usually high. On the other hand, shortest path methods, such as Dijkstra's algorithm which is based on priority queues, have been applied successfully to the solution of deterministic shortest-path Markov decision processes. Here, we propose an improved value iteration algorithm based on Dijkstra's algorithm for solving shortest path Markov decision processes. The experimental results on a stochastic shortest-path problem show the feasibility of our approach. © Springer Science+Business Media B.V. 2011.García Hernández, MDG.; Ruiz Pinales, J.; Onaindia De La Rivaherrera, E.; Aviña Cervantes, JG.; Ledesma Orozco, S.; Alvarado Mendez, E.; Reyes Ballesteros, A. (2012). New prioritized value iteration for Markov decision processes. Artificial Intelligence Review. 37(2):157-167. doi:10.1007/s10462-011-9224-zS157167372Agrawal S, Roth D (2002) Learning a sparse representation for object detection. In: Proceedings of the 7th European conference on computer vision. Copenhagen, Denmark, pp 1–15Bellman RE (1954) The theory of dynamic programming. Bull Amer Math Soc 60: 503–516Bellman RE (1957) Dynamic programming. Princeton University Press, New JerseyBertsekas DP (1995) Dynamic programming and optimal control. Athena Scientific, MassachusettsBhuma K, Goldsmith J (2003) Bidirectional LAO* algorithm. In: Proceedings of indian international conferences on artificial intelligence. p 980–992Blackwell D (1965) Discounted dynamic programming. Ann Math Stat 36: 226–235Bonet B, Geffner H (2003a) Faster heuristic search algorithms for planning with uncertainty and full feedback. In: Proceedings of the 18th international joint conference on artificial intelligence. Morgan Kaufmann, Acapulco, México, pp 1233–1238Bonet B, Geffner H (2003b) Labeled RTDP: improving the convergence of real-time dynamic programming. In: Proceedings of the international conference on automated planning and scheduling. Trento, Italy, pp 12–21Bonet B, Geffner H (2006) Learning depth-first search: a unified approach to heuristic search in deterministic and non-deterministic settings and its application to MDP. In: Proceedings of the 16th international conference on automated planning and scheduling. Cumbria, UKBoutilier C, Dean T, Hanks S (1999) Decision-theoretic planning: structural assumptions and computational leverage. J Artif Intell Res 11: 1–94Chang I, Soo H (2007) Simulation-based algorithms for Markov decision processes Communications and control engineering. Springer, LondonDai P, Goldsmith J (2007a) Faster dynamic programming for Markov decision processes. Technical report. Doctoral consortium, department of computer science and engineering. University of WashingtonDai P, Goldsmith J (2007b) Topological value iteration algorithm for Markov decision processes. In: Proceedings of the 20th international joint conference on artificial intelligence. Hyderabad, India, pp 1860–1865Dai P, Hansen EA (2007c) Prioritizing bellman backups without a priority queue. In: Proceedings of the 17th international conference on automated planning and scheduling, association for the advancement of artificial intelligence. Rhode Island, USA, pp 113–119Dibangoye JS, Chaib-draa B, Mouaddib A (2008) A Novel prioritization technique for solving Markov decision processes. In: Proceedings of the 21st international FLAIRS (The Florida Artificial Intelligence Research Society) conference, association for the advancement of artificial intelligence. Florida, USAFerguson D, Stentz A (2004) Focused propagation of MDPs for path planning. In: Proceedings of the 16th IEEE international conference on tools with artificial intelligence. pp 310–317Hansen EA, Zilberstein S (2001) LAO: a heuristic search algorithm that finds solutions with loops. Artif Intell 129: 35–62Hinderer K, Waldmann KH (2003) The critical discount factor for finite Markovian decision processes with an absorbing set. Math Methods Oper Res 57: 1–19Li L (2009) A unifying framework for computational reinforcement learning theory. PhD Thesis. The state university of New Jersey, New Brunswick. NJLittman ML, Dean TL, Kaelbling LP (1995) On the complexity of solving Markov decision problems.In: Proceedings of the 11th international conference on uncertainty in artificial intelligence. Montreal, Quebec pp 394–402McMahan HB, Gordon G (2005a) Fast exact planning in Markov decision processes. In: Proceedings of the 15th international conference on automated planning and scheduling. Monterey, CA, USAMcMahan HB, Gordon G (2005b) Generalizing Dijkstra’s algorithm and gaussian elimination for solving MDPs. Technical report, Carnegie Mellon University, PittsburghMeuleau N, Brafman R, Benazera E (2006) Stochastic over-subscription planning using hierarchies of MDPs. In: Proceedings of the 16th international conference on automated planning and scheduling. Cumbria, UK, pp 121–130Moore A, Atkeson C (1993) Prioritized sweeping: reinforcement learning with less data and less real time. Mach Learn 13: 103–130Puterman ML (1994) Markov decision processes. Wiley Editors, New YorkPuterman ML (2005) Markov decision processes. Wiley Inter Science Editors, New YorkRussell S (2005) Artificial intelligence: a modern approach. Making complex decisions (Ch-17), 2nd edn. Pearson Prentice Hill Ed., USAShani G, Brafman R, Shimony S (2008) Prioritizing point-based POMDP solvers. IEEE Trans Syst Man Cybern 38(6): 1592–1605Sniedovich M (2006) Dijkstra’s algorithm revisited: the dynamic programming connexion. Control Cybern 35: 599–620Sniedovich M (2010) Dynamic programming: foundations and principles, 2nd edn. Pure and Applied Mathematics Series, UKTijms HC (2003) A first course in stochastic models. Discrete-time Markov decision processes (Ch-6). Wiley Editors, UKVanderbei RJ (1996) Optimal sailing strategies. Statistics and operations research program, University of Princeton, USA ( http://www.orfe.princeton.edu/~rvdb/sail/sail.html )Vanderbei RJ (2008) Linear programming: foundations and extensions, 3rd edn. Springer, New YorkWingate D, Seppi KD (2005) Prioritization methods for accelerating MDP solvers. J Mach Learn Res 6: 851–88

Caracterización multi-ambiental de la fecha de floración en cultivares de Raigrás anual (L. multiflorum Lam.)

Publicado dentro de la Revista Argentina de Producción Animal, v 39 Suplemento 1 (2019).El importante número de cultivares de raigrás anual (RGA) presentes en el mercado, genera la necesidad de conocer tanto su potencial productivo como el comportamiento fenológico de los mismos, de manera de poder determinar que material se adapta mejor a cada sistema productivo. El objetivo del presente trabajo fue caracterizar el ciclo a floración de cultivares de Raigrás anual en diferentes localidades de la Argentina.EEA PergaminoFil: Ré, Alejo Esteban. Instituto Nacional de Tecnología Agropecuaria (INTA). Estación Experimental Agropecuaria Concepción del Uruguay; UruguayFil: Lavandera, Javier Eduardo. Instituto Nacional de Tecnología Agropecuaria (INTA). Estación Experimental Agropecuaria Pergamino. Forrajeras; ArgentinaFil: Barbera, Pablo. Instituto Nacional de Tecnología Agropecuaria (INTA). Estación Experimental Agropecuaria Mercedes (Corrientes); ArgentinaFil: Romero, Luis Alberto. Instituto Nacional de Tecnología Agropecuaria (INTA). Estación Experimental Agropecuaria Rafaela; ArgentinaFil: Gallego, Juan José. Instituto Nacional de Tecnología Agropecuaria (INTA). Estación Experimental Agropecuaria Valle Inferior de Río Negro; ArgentinaFil: Neira Zilli, Fernanda. Instituto Nacional de Tecnología Agropecuaria (INTA). Estación Experimental Agropecuaria Valle Inferior de Río Negro; ArgentinaFil: Méndez, Daniel Gustavo. Instituto Nacional de Tecnología Agropecuaria (INTA). Estación Experimental Agropecuaria General Villegas; Argentin

The landscape of Neandertal ancestry in present-day humans

Analyses of Neandertal genomes have revealed that Neandertals have contributed genetic variants to modern humans1–2. The antiquity of Neandertal gene flow into modern humans means that regions that derive from Neandertals in any one human today are usually less than a hundred kilobases in size. However, Neandertal haplotypes are also distinctive enough that several studies have been able to detect Neandertal ancestry at specific loci1,3–8. Here, we have systematically inferred Neandertal haplotypes in the genomes of 1,004 present-day humans12. Regions that harbor a high frequency of Neandertal alleles in modern humans are enriched for genes affecting keratin filaments suggesting that Neandertal alleles may have helped modern humans adapt to non-African environments. Neandertal alleles also continue to shape human biology, as we identify multiple Neandertal-derived alleles that confer risk for disease. We also identify regions of millions of base pairs that are nearly devoid of Neandertal ancestry and enriched in genes, implying selection to remove genetic material derived from Neandertals. Neandertal ancestry is significantly reduced in genes specifically expressed in testis, and there is an approximately 5-fold reduction of Neandertal ancestry on chromosome X, which is known to harbor a disproportionate fraction of male hybrid sterility genes20–22. These results suggest that part of the reduction in Neandertal ancestry near genes is due to Neandertal alleles that reduced fertility in males when moved to a modern human genetic background

Higher-order multipole amplitudes in charmonium radiative transitions

Using 24 million $\psi' \equiv \psi(2S)$ decays in CLEO-c, we have searched for higher multipole admixtures in electric-dipole-dominated radiative transitions in charmonia. We find good agreement between our data and theoretical predictions for magnetic quadrupole (M2) amplitudes in the transitions $\psi' \to \gamma \chi_{c1,2}$ and $\chi_{c1,2} \to \gamma J/\psi$, in striking contrast to some previous measurements. Let $b_2^J$ and $a_2^J$ denote the normalized M2 amplitudes in the respective aforementioned decays, where the superscript $J$ refers to the angular momentum of the $\chi_{cJ}$. By performing unbinned maximum likelihood fits to full five-parameter angular distributions, we determine the ratios $a_2^{J=1}/a_2^{J=2} = 0.67^{+0.19}_{-0.13}$ and $a_2^{J=1}/b_2^{J=1} = -2.27^{+0.57}_{-0.99}$, where the theoretical predictions are independent of the charmed quark magnetic moment and are $a_2^{J=1}/a_2^{J=2} = 0.676 \pm 0.071$ and $a_2^{J=1}/b_2^{J=1} = -2.27 \pm 0.16$.Comment: 32 pages, 7 figures, acceptance updat

Rapid tests and urine sampling techniques for the diagnosis of urinary tract infection (UTI) in children under five years: a systematic review

Background: Urinary tract infection (UTI) is one of the most common sources of infection in children under five. Prompt diagnosis and treatment is important to reduce the risk of renal scarring. Rapid, cost-effective, methods of UTI diagnosis are required as an alternative to culture. Methods: We conducted a systematic review to determine the diagnostic accuracy of rapid tests for detecting UTI in children under five years of age. Results: The evidence supports the use of dipstick positive for both leukocyte esterase and nitrite (pooled LR+ = 28.2, 95% CI: 17.3, 46.0) or microscopy positive for both pyuria and bacteriuria (pooled LR+ = 37.0, 95% CI: 11.0, 125.9) to rule in UTI. Similarly dipstick negative for both LE and nitrite (Pooled LR- = 0.20, 95% CI: 0.16, 0.26) or microscopy negative for both pyuria and bacteriuria (Pooled LR- = 0.11, 95% CI: 0.05, 0.23) can be used to rule out UTI. A test for glucose showed promise in potty-trained children. However, all studies were over 30 years old. Further evaluation of this test may be useful. Conclusion: Dipstick negative for both LE and nitrite or microscopic analysis negative for both pyuria and bacteriuria of a clean voided urine, bag, or nappy/pad specimen may reasonably be used to rule out UTI. These patients can then reasonably be excluded from further investigation, without the need for confirmatory culture. Similarly, combinations of positive tests could be used to rule in UTI, and trigger further investigation

Dalitz Plot Analysis of Ds to K+K-pi+

We perform a Dalitz plot analysis of the decay Ds to K+K-pi+ with the CLEO-c data set of 586/pb of e+e- collisions accumulated at sqrt(s) = 4.17 GeV. This corresponds to about 0.57 million D_s+D_s(*)- pairs from which we select 14400 candidates with a background of roughly 15%. In contrast to previous measurements we find good agreement with our data only by including an additional f_0(1370)pi+ contribution. We measure the magnitude, phase, and fit fraction of K*(892) K+, phi(1020)pi+, K0*(1430)K+, f_0(980)pi+, f_0(1710)pi+, and f_0(1370)pi+ contributions and limit the possible contributions of other KK and Kpi resonances that could appear in this decay.Comment: 21 Pages,available through http://www.lns.cornell.edu/public/CLNS/, submitted to PR

Search for D0 to p e- and D0 to pbar e+

Using data recorded by CLEO-c detector at CESR, we search for simultaneous baryon and lepton number violating decays of the D^0 meson, specifically, D^0 --> p-bar e^+, D^0-bar --> p-bar e^+, D^0 --> p e^- and D^0-bar --> p e^-. We set the following branching fraction upper limits: D^0 --> p-bar e^+ (D^0-bar --> p-bar e^+) p e^- (D^0-bar --> p e^-) < 1.2 * 10^{-5}, both at 90% confidence level.Comment: 10 pages, available through http://www.lns.cornell.edu/public/CLNS/, submitted to PRD. Comments: changed abstract, added reference for section 1, vertical axis in Fig.5 changed (starts from 1.5 rather than 2.0), fixed typo

Charmonium decays to gamma pi0, gamma eta, and gamma eta'

Using data acquired with the CLEO-c detector at the CESR e+e- collider, we measure branching fractions for J/psi, psi(2S), and psi(3770) decays to gamma pi0, gamma eta, and gamma eta'. Defining R_n = B[ psi(nS)-->gamma eta ]/B[ psi(nS)-->gamma eta' ], we obtain R_1 = (21.1 +- 0.9)% and, unexpectedly, an order of magnitude smaller limit, R_2 < 1.8% at 90% C.L. We also use J/psi-->gamma eta' events to determine branching fractions of improved precision for the five most copious eta' decay modes.Comment: 14 pages, available through http://www.lns.cornell.edu/public/CLNS/, published in Physical Review

Precision Measurement of the Mass of the h_c(1P1) State of Charmonium

A precision measurement of the mass of the h_c(1P1) state of charmonium has been made using a sample of 24.5 million psi(2S) events produced in e+e- annihilation at CESR. The reaction used was psi(2S) -> pi0 h_c, pi0 -> gamma gamma, h_c -> gamma eta_c, and the reaction products were detected in the CLEO-c detector. Data have been analyzed both for the inclusive reaction and for the exclusive reactions in which eta_c decays are reconstructed in fifteen hadronic decay channels. Consistent results are obtained in the two analyses. The averaged results of the present measurements are M(h_c)=3525.28+-0.19 (stat)+-0.12(syst) MeV, and B(psi(2S) -> pi0 h_c)xB(h_c -> gamma eta_c)= (4.19+-0.32+-0.45)x10^-4. Using the 3PJ centroid mass, Delta M_hf(1P)= - M(h_c) = +0.02+-0.19+-0.13 MeV.Comment: 9 pages, available through http://www.lns.cornell.edu/public/CLNS/, submitted to PR