Tight Performance Bounds for Approximate Modified Policy Iteration with Non-Stationary Policies

We consider approximate dynamic programming for the infinite-horizon stationary $\gamma$-discounted optimal control problem formalized by Markov Decision Processes. While in the exact case it is known that there always exists an optimal policy that is stationary, we show that when using value function approximation, looking for a non-stationary policy may lead to a better performance guarantee. We define a non-stationary variant of MPI that unifies a broad family of approximate DP algorithms of the literature. For this algorithm we provide an error propagation analysis in the form of a performance bound of the resulting policies that can improve the usual performance bound by a factor $O(1-\gamma)$, which is significant when the discount factor $\gamma$ is close to 1. Doing so, our approach unifies recent results for Value and Policy Iteration. Furthermore, we show, by constructing a specific deterministic MDP, that our performance guarantee is tight

On the Use of Non-Stationary Policies for Stationary Infinite-Horizon Markov Decision Processes

We consider infinite-horizon stationary $\gamma$-discounted Markov Decision Processes, for which it is known that there exists a stationary optimal policy. Using Value and Policy Iteration with some error $\epsilon$ at each iteration, it is well-known that one can compute stationary policies that are $\frac{2\gamma}{(1-\gamma)^2}\epsilon$-optimal. After arguing that this guarantee is tight, we develop variations of Value and Policy Iteration for computing non-stationary policies that can be up to $\frac{2\gamma}{1-\gamma}\epsilon$-optimal, which constitutes a significant improvement in the usual situation when $\gamma$ is close to 1. Surprisingly, this shows that the problem of "computing near-optimal non-stationary policies" is much simpler than that of "computing near-optimal stationary policies"

Monterey County World AIDS Day-fundraiser and community gathering

World AIDS Day is observed in over twenty countries annually on December 1st. On that day, millions of people gather to raise awareness about AIDS. My capstone involves working with a local HIV/AIDS service provider, called John XXIII, to plan a fundraiser and community gathering, in observance of World AIDS Day. The goal of these events was to raise awareness and money for HIV/AIDS treatment and prevention services in Monterey County

Obywatelstwo i narodowość w językowym obrazie świata młodzieży

In the undertaken study, the understanding of the concepts of citizenship and nationality by the Polish young people, I assume that these categories are essential in terms of shaping the national identity of the individual. The system of concepts that define the personality of a human, the ability to reflect on their content, correctness, common connection, allows individuals to unite into a certain community. Thus, the analysis of linguistic ways of describing non-linguistic reality is also associated with the description of the mental representations of the individual, and therefore the ways – in cognitive processes – to relate to the world, to make an image of the world, to create knowledge about the world

Examining Patterns and Predictors of Response to Mathematics Intervention

Concerns about low mathematics achievement have created a push to increase national mathematics proficiency levels through research and policy. Efforts have largely focused on early mathematics interventions for students with or at risk for mathematics learning difficulties implemented within multi-tiered response to intervention (RTI) frameworks. Within these models, important educational decisions are based upon instructional response, making meaningful categorization of student responsiveness to intervention paramount. Across research and practice, intervention outcomes are typically thought of as a binary, with students considered either responsive or non-responsive to intervention. However, defining and categorizing responsiveness in more complex ways may reveal important differences between subgroups of students who exhibit distinct patterns of responsiveness over time. The present study explored patterns of response to an early mathematics intervention using data from the ROOTS Efficacy Project. Participants included kindergarten students at risk for mathematics difficulties who were randomly assigned to the ROOTS intervention condition (n = 880). Results of a latent profile analysis indicated that variability in response to a generally effective intervention was best captured by a more complex categorization framework encompassing four distinct response profiles: a moderate-risk, mildly responsive group; a moderate-risk, delayed response group, a high-risk, strongly responsive group; and a lower-risk, non-responsive group. Membership in each response profile group was predicted by pre-intervention performance on measures of both early mathematics and cognitive skills (visual-spatial, fluid reasoning, and working memory). On average, students with lower initial math skill and cognitive performance demonstrated stronger intervention response. Implications for future research and practice are discussed

Optimal price subsidies for appropriate malaria testing and treatment behaviour.

BACKGROUND: Malaria continues to be a serious public health problem particularly in Africa. Many people infected with malaria do not access effective treatment due to high price. At the same time many individuals receiving malaria drugs do not suffer from malaria because of the common practice of presumptive diagnosis. A global subsidy on artemisinin-based combination therapy (ACT) has recently been suggested to increase access to the most effective malaria treatment. METHODS: Following the recommendation by World Health Organization that parasitological testing should be performed before treatment and ACT prescribed to confirmed cases only, it is investigated in this paper if a subsidy on malaria rapid diagnostic tests (RDTs) should be incorporated. A model is developed consisting of a representative individual with fever suspected to be malaria, seeking care at a specialized drug shop where RDTs, ACT medicines, and cheap, less effective anti-malarials are sold. Assuming that the individual has certain beliefs of the accuracy of the RDT and the probability that the fever is malaria, the model predicts the diagnosis-treatment behaviour of the individual. Subsidies on RDTs and ACT are introduced to incentivize appropriate behaviour: choose an RDT before treatment and purchase ACT only if the test is positive. RESULTS: Solving the model numerically suggests that a combined subsidy on both RDT and ACT is cost minimizing and improves diagnosis-treatment behaviour of individuals. For certain beliefs, such as low trust in RDT accuracy and strong belief that a fever is malaria, subsidization is not sufficient to incentivize appropriate behaviour. CONCLUSIONS: A combined subsidy on both RDT and ACT rather than a single subsidy is likely required to improve diagnosis-treatment behaviour among individuals seeking care for malaria in the private sector

Bali - raj wymyślony?

Bali – raj wymyślony?, w: Kulturowe uwarunkowania rozwoju współczesnej Azji, red. Michał Gołoś, Joanna Marszałek-Kawa, Wydawnictwo Adam Marszałek, Toruń 2016, s. 78-95