Optimistic Planning for Markov Decision Processes

International audienceThe reinforcement learning community has recently intensified its interest in online planning methods, due to their relative independence on the state space size. However, tight near-optimality guarantees are not yet available for the general case of stochastic Markov decision processes and closed-loop, state-dependent planning policies. We therefore consider an algorithm related to AO* that optimistically explores a tree representation of the space of closed-loop policies, and we analyze the near-optimality of the action it returns after n tree node expansions. While this optimistic planning requires a finite number of actions and possible next states for each transition, its asymptotic performance does not depend directly on these numbers, but only on the subset of nodes that significantly impact near-optimal policies. We characterize this set by introducing a novel measure of problem complexity, called the near-optimality exponent. Specializing the exponent and performance bound for some interesting classes of MDPs illustrates the algorithm works better when there are fewer near-optimal policies and less uniform transition probabilities

A (2003). The children's Yale-Brown obsessive-compulsive scale: Item structure in an outpatient setting

The Children's Yale-Brown Obsessive-Compulsive Scale (CY-BOCS) is one of the most popular measures of symptom severity for childhood obsessive-compulsive disorder (OCD). This study describes the factor structure of the CY-BOCS. A total of 233 children diagnosed with OCD were evaluated with the CY-BOCS. The results indicated that 2 alternate 2-factor solutions (obsessions and compulsions; severity and disturbance) fit the data set best. The results also suggested that items assessing resistance to obsessions and compulsions provide unreliable evaluations of these symptoms in children with OCD. Recommendations for modifications to the CY-BOCS as well as clinical applications and for future research with the measure are made

The Children\u27s Yale-Brown Obsessive-Compulsive Scale: Item Structure in an Outpatient Setting.

The Children\u27s Yale-Brown Obsessive-Compulsive Scale (CY-BOCS) is one of the most popular measures of symptom severity for childhood obsessive-compulsive disorder (OCD). This study describes the factor structure of the CY-BOCS. A total of 233 children diagnosed with OCD were evaluated with the CY-BOCS. The results indicated that 2 alternate 2-factor solutions (obsessions and compulsions; severity and disturbance) fit the data set best. The results also suggested that items assessing resistance to obsessions and compulsions provide unreliable evaluations of these symptoms in children with OCD. Recommendations for modifications to the CY-BOCS as well as clinical applications and for future research with the measure are made

Overvalued ideas and their impact on treatment outcome Idéias supervalorizadas e seu impacto no resultado do tratamento

BACKGROUND: The presence of overvalued ideas (OVI) in patients with obsessive-compulsive disorder (OCD) has been theoretically linked to poorer treatment outcome. However, to date there have not been any quantitative measures of overvalued ideas. Therefore, the relationship between OVI and outcome has been primarily hypothetical. Rudimentary assessments have been attempted by asking patients to rate their strength of belief from 1 to 10, clinically rating the fixity of beliefs from 1 to 5, and rating patients on item 11 (insight) on the Yale-Brown Obsessive Compulsive Scale (Y-BOCS). The purpose of this study was to determine the relationship between OVI, degree of severity of OCD symptoms, and improvement. METHOD: Twenty patients with OCD participated in the study. All patients were treated with six days a week of 90 minutes of exposure and response prevention (ERP) and selective serotonin re-uptake inhibitors. No changes in medication were made during the course of treatment and all patients had been on their respective medications for at least three months prior to entering the study. Assessment scales consisted of the Overvalued Ideas Scale (OVIS), Y-BOCS, Beck Depression Inventory, and the Beck Anxiety Inventory. RESULTS: The results indicated that overvalued ideas did not necessarily correlate with severity of symptoms on the Y-BOCS. The higher the score on the OVIS, the less change on the Y-BOCS score. CONCLUSION: Overvalued ideas appear to be an important predictor of poor outcome.<br>INTRODUÇÃO: A existência de idéias superestimadas (IS) em pacientes com transtorno obsessivo-compulsivo (TOC) vem sendo teoricamente relacionada a resultado insatisfatório do tratamento. Como até o momento não existem medidas quantitativas das idéias superestimadas, a relação entre IS e resultado fica restrita ao campo hipotético. Houve algumas tentativas de avaliações simples. Era pedido aos pacientes que avaliassem a intensidade de suas convicções, em uma escala de 1 a 10. A seguir, fazia-se a avaliação clínica da fixidez das convicções de 1 a 5, e os pacientes eram classificados segundo o item 11 (insight) da Escala Yale-Brown de Sintomas Obsessivo-compulsivos (Y-BOCS). A proposta deste estudo era determinar a relação existente entre IS, gravidade dos sintomas do TOC e melhora. MÉTODO: Vinte pacientes com DOC participaram do estudo. Todos eles foram submetidos ao tratamento de prevenção de exposição e reação (PER), durante seis dias por semana, com sessões de 90 minutos, e medicados com inibidores seletivos de recaptação de serotonina. Os medicamentos não foram alterados durante o curso do tratamento. Antes de entrar no estudo, todos os pacientes já recebiam tratamento medicamentoso há, pelo menos, três meses. As escalas de avaliação usadas foram Escala de Idéias Superestimadas (OVIS), Y-BOCS, Inventário Beck para Depressão e Inventário Beck para Ansiedade. RESULTADOS: Os resultados indicaram que idéias superestimadas não se correlacionavam necessariamente com a gravidade dos sintomas, medidos pela Y-BOCS. Quanto maior o grau medido pela OVIS, menor era a variação na Y-BOCS. CONCLUSÃO: Idéias superestimadas parecem ser um importante fator de predição de um resultado insatisfatório para o tratamento