Markov abstractions for PAC reinforcement learning in non-Markov decision processes

Our work aims at developing reinforcement learning algorithms that do not rely on the Markov assumption. We consider the class of Non-Markov Decision Processes where histories can be abstracted into a finite set of states while preserving the dynamics. We call it a Markov abstraction since it induces a Markov Decision Process over a set of states that encode the non-Markov dynamics. This phenomenon underlies the recently introduced Regular Decision Processes (as well as POMDPs where only a finite number of belief states is reachable). In all such kinds of decision process, an agent that uses a Markov abstraction can rely on the Markov property to achieve optimal behaviour. We show that Markov abstractions can be learned during reinforcement learning. Our approach combines automata learning and classic reinforcement learning. For these two tasks, standard algorithms can be employed. We show that our approach has PAC guarantees when the employed algorithms have PAC guarantees, and we also provide an experimental evaluation

Post-partum follicular dynamics in beef cows calving during spring and autumn in southern Brazil.

Ovarian activity early post-partum in beef cows with intermediate body condition scores that calved during spring and autumn and treated with either 48 h of temporary weaning or exogenous hormones was investigated. Calving cows were given body condition scores and their ovaries were ultrasonographically scanned daily starting on day ten postpartum. The number and size of the follicles were recorded. Upon detection of a dominant follicle (>9 mm), the animals were distributed to different treatments. Over 80% of the animals (41/49) in both seasons presented a dominant follicle during the second or third week post-partum. The percentage of cows ovulating within seven days after treatment varied from 30% (3/10) for control cows to 60% (6/10) for MAP+GnRH treated cows for both spring and autumn calving cows. A reduction of 16% and 19% in body condition score was observed during the post-partum period studied for both spring and autumn calving cows, respectively. The decrease in body condition score was accompanied by a reduction in the follicular population of 43% during the fifth week post-partum only in those calving during autumn. In the spring calving cows, no change was detected in the follicular population despite the decrease in body condition score. Irrespective of the differences in environmental conditions between the two breeding seasons, cows present large follicles in their ovaries that are capable of responding to hormonal treatments, during the early post-partum period.Doc 1. Disponível em: . Acesso em: 27 ago. 2018

Fetal development and blood hematological-biochemical parameters in Campeiro and Pantaneiro foals.

Made available in DSpace on 2019-01-02T23:37:38Z (GMT). No. of bitstreams: 1 2018Vieiraetal..pdf: 293128 bytes, checksum: 55472a23573b40b9efd496c73b492ee6 (MD5) Previous issue date: 2019-01-02bitstream/item/189575/1/2018-Vieira-et-al.-.pd

Heat tolerance in naturalised cattle in Brazil: physical factors.

Made available in DSpace on 2018-06-06T00:58:35Z (GMT). No. of bitstreams: 1 ID278101.pdf: 88655 bytes, checksum: ad66fec8695cc2d52bc475f0eb728fa3 (MD5) Previous issue date: 2007-01-0

SmartIX: A database indexing agent based on reinforcement learning

Configuring databases for efficient querying is a complex task, often carried out by a database administrator. Solving the problem of building indexes that truly optimize database access requires a substantial amount of database and domain knowledge, the lack of which often results in wasted space and memory for irrelevant indexes, possibly jeopardizing database performance for querying and certainly degrading performance for updating. In this paper, we develop the SMARTIX architecture to solve the problem of automatically indexing a database by using reinforcement learning to optimize queries by indexing data throughout the lifetime of a database. We train and evaluate SMARTIX performance using TPC-H, a standard, and scalable database benchmark. Our empirical evaluation shows that SMARTIX converges to indexing configurations with superior performance compared to standard baselines we define and other reinforcement learning methods used in related work