229 research outputs found

    Multi-step Reinforcement Learning: A Unifying Algorithm

    Full text link
    Unifying seemingly disparate algorithmic ideas to produce better performing algorithms has been a longstanding goal in reinforcement learning. As a primary example, TD(λ\lambda) elegantly unifies one-step TD prediction with Monte Carlo methods through the use of eligibility traces and the trace-decay parameter λ\lambda. Currently, there are a multitude of algorithms that can be used to perform TD control, including Sarsa, QQ-learning, and Expected Sarsa. These methods are often studied in the one-step case, but they can be extended across multiple time steps to achieve better performance. Each of these algorithms is seemingly distinct, and no one dominates the others for all problems. In this paper, we study a new multi-step action-value algorithm called Q(σ)Q(\sigma) which unifies and generalizes these existing algorithms, while subsuming them as special cases. A new parameter, σ\sigma, is introduced to allow the degree of sampling performed by the algorithm at each step during its backup to be continuously varied, with Sarsa existing at one extreme (full sampling), and Expected Sarsa existing at the other (pure expectation). Q(σ)Q(\sigma) is generally applicable to both on- and off-policy learning, but in this work we focus on experiments in the on-policy case. Our results show that an intermediate value of σ\sigma, which results in a mixture of the existing algorithms, performs better than either extreme. The mixture can also be varied dynamically which can result in even greater performance.Comment: Appeared at the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18

    Understanding the Extent, Composition, and Characteristics of the Poor

    Get PDF
    Probing deeper into the poverty picture, this Policy Note examines the extent, composition, and characteristics of the poor, using various rounds of the Family Income and Expenditure Survey (FIES) and Annual Poverty Indicator Survey (APIS) as panel data sets. Understanding these aspects could provide insights that may guide the government in formulating specific types of interventions for different groups of households, especially the chronic and transient poor.Philippines, chronic poor, transient poor, panel data

    Corporate debt, firm size and financial fragility in emerging markets

    Get PDF
    The post-Global Financial Crisis period shows a surge in corporate leverage in emerging markets and a number of countries with deteriorated corporate financial fragility indicators (Altman's Z-score). Firm size plays a critical role in the relationship between leverage, firm fragility and exchange rate movements in emerging markets. While the relationship between firm-leverage and distress scores varies over time, the relationship between firm size and corporate vulnerability is relatively time-invariant. All else equal, large firms in emerging markets are more financially vulnerable and also systemically important. Consistent with the granular origins of aggregate fluctuations in Gabaix (2011), idiosyncratic shocks to the sales growth of large firms are positively and significantly correlated with GDP growth in our emerging markets sample. Relatedly, the negative impact of exchange rate shocks has a more acute impact on the sales growth of the more highly levered large firms

    Needs and Design Assessment for Corn Production Enterprise in Selected Agrarian Reform Communities of Cagayan de Oro City and Lanao del Norte, Southern Philippines

    Get PDF
    The needs and design assessment was conducted in the Agrarian Reform Communities of Cagayan de Oro City and Lanao del Norte. It aimed to identify performance requirements, knowledge, skills, and abilities of the Agrarian Reform Beneficiaries Organizations (ARBO), as well as the technology and facility needs for a corn production enterprise. Among the ARBOs assessed were: Pagalungan and Tagpangi Tribal and Settlers Association of Women Agrarian Reform Beneficiaries Cooperative, Taglimao Farmers Multi-purpose Cooperative, Indahag Agrarian Reform Community Multipurpose Cooperative, Taguanao Agrarian Reform Community Multipurpose Cooperative and Palambu Cooperative. The needs assessment revealed that the ARBOs still need technologies, as well as facilities and machineries for corn production and marketing. The ARCs offer biophysical condition suitable for corn production. They have natural assets that would favor corn production as the main economic activity. Corn is an appropriate commodity for the ARCCESS project in the ARCs mentioned. The farmers in the ARCs have extensive experience and training related to corn production. The ARBOs have other needs, such as capability development, common service facility on pre- and post-harvest facilities, project monitoring and enterprise development, and financial management and assistance. With the provision of the aforementioned recommended activities and facilities, this should result in increased production, improved quality of products, increased income and realized more profits; thereby improving their standard of living. This will eventually lead to economic growth and development of the community and to the economy as a whole

    'Diverse mobilities': second-generation Greek-Germans engage with the homeland as children and as adults

    Get PDF
    This paper is about the children of Greek labour migrants in Germany. We focus on two life-stages of ‘return’ for this second generation: as young children brought to Greece on holidays or sent back for longer periods, and as young adults exercising an independent ‘return’ migration. We draw both on literature and on our own field interviews with 50 first- and second-generation Greek-Germans. We find the practise of sending young children back to Greece to have been surprisingly widespread yet little documented. Adult relocation to the parental homeland takes place for five reasons: (i) a ‘search for self’; (ii) attraction of the Greek way of life; (iii) the actualisation of the ‘family narrative of return’ by the second, rather than the first, generation; (iv) life-stage events such as going to university or marrying a Greek; (v) escape from a traumatic event or oppressive family situation. Yet the return often brings difficulties, disillusionment, identity reappraisal, and a re-evaluation of the German context

    UPLC-MS analysis of cannabis sativa using tetrahydrocannabinol (THC), cannabidiol (CBD), and tetrahydrocannabinolic acid (THCA) as marker compounds: inhibition of breast cancer cell survival and progression

    Get PDF
    Cannabis sativa L. extracts were characterized by ultra performance liquid chromatography-mass spectrometry (UPLC-MS) using tetrahydrocannabinol (THC), cannabidiol (CBD), and tetrahydrocannabinolic acid (THCA) as marker compounds. The inhibitory effects of various extracts were determined on the survival and progression of highly metastatic breast cancer cells. A higher amount of CBD was found in the dichloromethane extract, and this was found to be effective in inhibiting breast cancer cell growth in vitro and in angiogenesis. Collectively, it may be concluded that CBD, THC, and THCA in the African variety of C. sativa can be used as marker compounds in UPLC-MS analysis. The ability of the plant to inhibit breast cancer cell survival and progression may affirm the traditional use of the drug as an anticancer agent.The National Research Foundation and DST-IKS Based Technology, South Africa.https://journals.sagepub.com/home/npxpm2020Chemistr

    Ward-based Goal-Directed Fluid Therapy (GDFT) in Acute Pancreatitis (GAP) trial: study protocol for a feasibility randomised controlled trial

    Get PDF
    IntroductionAcute pancreatitis is an inflammatory disease of the pancreas with high risk of developing multiorgan failure and death. There are no effective pharmacological interventions used in current clinical practice. Maintaining fluid and electrolyte balance is the mainstay of supportive management. Goal-directed fluid therapy (GDFT) has been shown to decrease morbidity and mortality in surgical conditions with systemic inflammatory response. There is currently no randomised controlled trial (RCT) investigating the role of GDFT based on cardiac output parameters in patients with acute pancreatitis in the ward setting. A feasibility trial was designed to determine patient and clinician support for recruitment into an RCT of ward-based GDFT in acute pancreatitis, adherence to a GDFT protocol, safety, participant withdrawal, and to determine appropriate endpoints for a subsequent larger trial to evaluate efficacy.Methods and analysisThe GDFT in Acute Pancreatitis trial is a prospective two-centre feasibility RCT. Eligible adults admitted with new onset of acute pancreatitis will be enrolled and randomised into ward-based GDFT (n=25) or standard fluid therapy (n=25) within 6 hours from the diagnosis and continuing for the following 48 hours. Cardiac output parameters will be monitored with a non-invasive device (Cheetah NICOM; Cheetah Medical). The intervention group will consist of a protocolised GDFT approach consisting of stroke volume optimisation with crystalloid fluid boluses, while the control group will receive standard care fluid therapy as advised by the clinical team. The primary endpoint is feasibility. Secondary endpoints will include safety of the intervention, complications, mortality, admission to intensive care unit, cost and quality of life.Ethics and disseminationEthics approval was granted by the London Central Research Ethics Committee (17/LO/1235, project ID: 221872). The results of this trial will be presented to international conference with interest in general surgery and acute care and published in a peer-reviewed journal.Trial registration numberISRCTN36077283.</jats:sec
    • 

    corecore