Curriculum Learning for Cumulative Return Maximization

Curriculum learning has been successfully used in reinforcement learning to accelerate the learning process, through knowledge transfer between tasks of increasing complexity. Critical tasks, in which suboptimal exploratory actions must be minimized, can benefit from curriculum learning, and its ability to shape exploration through transfer. We propose a task sequencing algorithm maximizing the cumulative return, that is, the return obtained by the agent across all the learning episodes. By maximizing the cumulative return, the agent not only aims at achieving high rewards as fast as possible, but also at doing so while limiting suboptimal actions. We experimentally compare our task sequencing algorithm to several popular metaheuristic algorithms for combinatorial optimization, and show that it achieves significantly better performance on the problem of cumulative return maximization. Furthermore, we validate our algorithm on a critical task, optimizing a home controller for a micro energy grid

An Optimization Framework for Task Sequencing in Curriculum Learning

Curriculum learning in reinforcement learning is used to shape exploration by presenting the agent with increasingly complex tasks. The idea of curriculum learning has been largely applied in both animal training and pedagogy. In reinforcement learning, all previous task sequencing methods have shaped exploration with the objective of reducing the time to reach a given performance level. We propose novel uses of curriculum learning, which arise from choosing different objective functions. Furthermore, we define a general optimization framework for task sequencing and evaluate the performance of popular metaheuristic search methods on several tasks. We show that curriculum learning can be successfully used to: improve the initial performance, take fewer suboptimal actions during exploration, and discover better policies

A New Highly Conserved Antibiotic Sensing/Resistance Pathway in Firmicutes Involves an ABC Transporter Interplaying with a Signal Transduction System

Signal transduction systems and ABC transporters often contribute jointly to adaptive bacterial responses to environmental changes. In Bacillus subtilis, three such pairs are involved in responses to antibiotics: BceRSAB, YvcPQRS and YxdJKLM. They are characterized by a histidine kinase belonging to the intramembrane sensing kinase family and by a translocator possessing an unusually large extracytoplasmic loop. It was established here using a phylogenomic approach that systems of this kind are specific but widespread in Firmicutes, where they originated. The present phylogenetic analyses brought to light a highly dynamic evolutionary history involving numerous horizontal gene transfers, duplications and lost events, leading to a great variety of Bce-like repertories in members of this bacterial phylum. Based on these phylogenetic analyses, it was proposed to subdivide the Bce-like modules into six well-defined subfamilies. Functional studies were performed on members of subfamily IV comprising BceRSAB from B. subtilis, the expression of which was found to require the signal transduction system as well as the ABC transporter itself. The present results suggest, for the members of this subfamily, the occurrence of interactions between one component of each partner, the kinase and the corresponding translocator. At functional and/or structural levels, bacitracin dependent expression of bceAB and bacitracin resistance processes require the presence of the BceB translocator loop. Some other members of subfamily IV were also found to participate in bacitracin resistance processes. Taken together our study suggests that this regulatory mechanism might constitute an important common antibiotic resistance mechanism in Firmicutes. [Supplemental material is available online at http://www.genome.org.

I cittadini e il sistema sanitario: c'è bisogno di fiducia?

Obiettivi. La fiducia negli altri ha una grande rilevanza nella vita economica, politica e sociale di un paese: infatti, laddove la fiducia reciproca è elevata, la società funziona meglio, è più produttiva, più cooperativa, più coesa, meno diffusi sono i comportamenti opportunistici e più ridotto è il livello della corruzione. Alla luce dei dati più recenti, ci proponiamo di indagare la fiducia come componente fondamentale della cura, che si costruisce nel tempo attraverso i contatti ripetuti tra il cittadino e i servizi sanitari. Metodi. Sono stati analizzati i principali studi nazionali-internazionali che dal 2003 ad oggi hanno posto al centro il tema della fiducia dei cittadini nelle istituzioni sociali, sanitarie e in maniera più dettagliata si sono interrogati sulla fiducia dopo un’esperienza di cura. Risultati. La proporzione di italiani che ha fiducia nei confronti delle istituzioni sociali (54%) risulta essere una delle più basse tra i paesi dell’area OECD, mostrando una forbice di disuguaglianza tra centro Nord e Sud dell’Italia (41,3% vs 26,3%). Risulta, invece, largamente apprezzata la professionalità e la competenza di medici e infermieri per il 64,2% degli intervistati. Aumenta inoltre la percentuale degli italiani soddisfatti del Servizio Sanitario Nazionale se si chiede di esprimersi sulla base dell’esperienza diretta e non secondo una generica opinione. Conclusioni. La fiducia è un valore che richiede alle istituzioni di vigilare sull’equilibrio fragile tra delega e valutazione, in particolare valorizzando le buone pratiche e la comunicazione trasparente dei dati, indagando l’esperienza dei pazienti e creando una partnership tra tutti gli attori coinvolti. Ulteriori approfondimenti sono necessari per comprendere quali siano i più efficaci metodi di costruzione di partnership all’interno delle organizzazioni sanitarie

A novel optimization perspective to the problem of designing sequences of tasks in a reinforcement learning framework

Training agents over sequences of tasks is often employed in deep reinforcement learning to let the agents progress more quickly towards better behaviours. This problem, known as curriculum learning, has been mainly tackled in the literature by numerical methods based on enumeration strategies, which, however, can handle only small size problems. In this work, we define a new optimization perspective to the curriculum learning problem with the aim of developing efficient solution methods for solving complex reinforcement learning tasks. Specifically, we show how the curriculum learning problem can be viewed as an optimization problem with a nonsmooth and nonconvex objective function and with an integer feasible region. We reformulate it by defining a grey-box function that includes a suitable scheduling problem. Numerical results on a benchmark environment in the reinforcement learning community show the effectiveness of the proposed approaches in reaching better performance also on large problems

Exploring Continuity of Care: The Patient Voice

Background: Oncology is one of the priorities of public health given its high incidence and prevalence. In 2010 there were 8 million deaths caused by cancer. In Italy neoplastic disease is the second highest cause of death. Considering these data, we need to guarantee appropriate, quality healthcare responses. In order to monitor the quality of cancer care pathways, we intend to explore continuity of care from patients\u2019 perspective, identifying the dimensions that define continuity. Methods: Semi-structured interviews with 30 cancer patients (4 colorectal and 26 breast cancer) who received treatment at different service points of the Area Vasta Network (AVR), Italy. To identify the macro-categories of continuity, all interviews were transcribed and analysed using framework analysis, assisted by a computer software package for analysis of qualitative data (N-VIVO 10). Simultaneously, a literature review was carried out using the Pubmed database to examine the continuity of care measures validated. Results: From the narratives of 30 patients, different continuity of care\u2019s dimensions emerged, for example the presence of a professional who knows the patient\u2019s illness history and takes him/her from initial diagnosis to followup care, guaranteeing him/her accurate information; a multi-professional team. The same aspects result central and transversal to 5 questionnaires identified in literature. Conclusions: The analysis allows us to identify 3 central and transversal dimensions of continuity of care: informational, organizational and relational; confirming the continuity of care model produced by Haggerty et al. It follows that in order to cater to the needs of cancer patients; we need to focus simultaneously on these three dimensions along the cancer care pathway. In line with these results, we are developing the first Italian patients experience continuity of care tool

Health Literacy e Shared Decision Making in Oncologia. Prospettive di studio nell'Ausl di Ravenna

Un crescente numero di ricerche mostra che la qualit\ue0 delle cure erogate aumenta quando i pazienti sono ben informati e giocano un ruolo attivo nella gestione del proprio percorso di salute. L\u2019interesse alla partecipazione del paziente al processo decisionale sul trattamento di cura si \ue8 spostato ben oltre il consenso informato per includere pi\uf9 ampi principi di alfabetizzazione sanitaria (Health Literacy), autonomia e controllo del percorso assistenziale: un processo decisionale condiviso (Shared Decision Making) \ue8 visto come un meccanismo per ridurre l\u2019asimmetria informativa tra paziente e medico, aumentare la consapevolezza della persona, il suo senso di autonomia e/o di controllo sulle decisioni di trattamento che lo riguarda. A partire da queste considerazioni, l\u2019Azienda Usl di Ravenna, in collaborazione con il Dipartimento di medicina e sanit\ue0 pubblica dell\u2019Universit\ue0 di Bologna, l\u2019Istituto di ricerca e cura a carattere scientifico (Irccs) di Meldola e l\u2019Agenzia Nazionale per i Servizi Sanitari Regionali, ha avviato il progetto di ricerca \u201cO.P.T.I.On\u201d (Opportunit\ue0 Per Il Trattamento In Oncologia) che si pone l\u2019obiettivo di analizzare la comunicazione medico-paziente in ambito oncologico, il grado di supporto dato al paziente nel definire e decidere il tipo di trattamento da seguire, con uno speciale focus sull\u2019opportunit\ue0 di ricevere un secondo parere e, infine, la soddisfazione del paziente sul percorso di cura intrapreso

Patient engagement in health care. New perspectives in oncology. [La partecipazione della persona al percorso di cura. Nuove prospettive in oncologia]

During the last decade, an increasing number of studies has been focused on patient engagement programs through which patients can be effective and informed managers of their health and health care. Based on our recent review, studies examining the nature of patient participation in service planning have found that most activities are limited to various forms of consultation, rather than the interactive partnerships advocated for treatment decision-making. Despite this clear evidence, many people want to play an active role in their own healthcare. They want to know how to protect and improve their health when they are well; and when they are ill, they want information about treatment options and likely outcomes. Consistent with these considerations, this paper intends to underline the importance to engage patients, to describe some different interventions to make patient effective informed; and, finally, to show some pilot projects from around the world, with a special focus on cancer care service of Ontario

Continuity of care of cancer patients. Analysis of theoretical models and survey tools of continuity of care in people with a cancer diagnosis

Background The Italian Ministry of Health declared oncology a priority and stressed the importance of ensuring continuity and integration in cancer care pathways. In order to monitor the quality of cancer care pathways, we need to explore patients\u2019 experience of the continuity of care, identifying the dimensions that define continuity Methods We found 886 relevant articles in the Pubmed database from 1987 to 5 November 2013. The search strategy for the electronic database was defined using the Population, Intervention, Comparison and Outcome(s) framework (PICO) to identify keywords. Two researchers independently reviewed records identified through the search strategy, analyzing continuity dimensions, specificity and/or transversal domains Results We selected 20 articles that measure the patients\u2019 experience of continuity of care: 7 articles including 5 questionnaires [Questionnaire by King et al. 2008; Cancer care coordination Questionnaire (Cccq); Patient Continuity of Care Questionnaire (Pccq); Medical Care Questionnaire (Mcq); Continuity and Coordination of Care Questionnaire (CCCQ)]; 6 articles evaluating the relationship between patient and his/her physician (the same across the care pathway) in terms of frequency and/or dispersion; 6 articles considering one subscale of larger scales designed to evaluate the generic cancer care service patient experience; 1 revealing four organizational indicators of care pathway continuity / discontinuity Conclusion We traced 3 transversal dimensions across the individual analyses: informational, organizational, relational continuity. It follows that in order to cater to the needs of cancer patients, we need to simultaneously focus on these three dimensions along the cancer care pathway. In line with these results, we promoted the \u201cR.In.Cu.ORAM.i\u201d study (Networks for Integrated Treatment of colorectal and breast cancer), in Area Vasta Romagna Area (Italy), and developed a continuity of care patient-experience continuity too