Adaptive Reward-Free Exploration

Reward-free exploration is a reinforcement learning setting studied by Jin et al. (2020), who address it by running several algorithms with regret guarantees in parallel. In our work, we instead give a more natural adaptive approach for reward-free exploration which directly reduces upper bounds on the maximum MDP estimation error. We show that, interestingly, our reward-free UCRL algorithm can be seen as a variant of an algorithm of Fiechter from 1994, originally proposed for a different objective that we call best-policy identification. We prove that RF-UCRL needs of order $({SAH^4}/{\varepsilon^2})(\log(1/\delta) + S)$ episodes to output, with probability $1-\delta$, an $\varepsilon$-approximation of the optimal policy for any reward function. This bound improves over existing sample-complexity bounds in both the small $\varepsilon$ and the small $\delta$ regimes. We further investigate the relative complexities of reward-free exploration and best-policy identification

Improved Sample Complexity for Incremental Autonomous Exploration in MDPs

We investigate the exploration of an unknown environment when no reward function is provided. Building on the incremental exploration setting introduced by Lim and Auer [1], we define the objective of learning the set of $\epsilon$-optimal goal-conditioned policies attaining all states that are incrementally reachable within $L$ steps (in expectation) from a reference state $s_0$. In this paper, we introduce a novel model-based approach that interleaves discovering new states from $s_0$ and improving the accuracy of a model estimate that is used to compute goal-conditioned policies to reach newly discovered states. The resulting algorithm, DisCo, achieves a sample complexity scaling as $\tilde{O}(L^5 S_{L+\epsilon} \Gamma_{L+\epsilon} A \epsilon^{-2})$, where $A$ is the number of actions, $S_{L+\epsilon}$ is the number of states that are incrementally reachable from $s_0$ in $L+\epsilon$ steps, and $\Gamma_{L+\epsilon}$ is the branching factor of the dynamics over such states. This improves over the algorithm proposed in [1] in both $\epsilon$ and $L$ at the cost of an extra $\Gamma_{L+\epsilon}$ factor, which is small in most environments of interest. Furthermore, DisCo is the first algorithm that can return an $\epsilon/c_{\min}$-optimal policy for any cost-sensitive shortest-path problem defined on the $L$-reachable states with minimum cost $c_{\min}$. Finally, we report preliminary empirical results confirming our theoretical findings.Comment: NeurIPS 202

Supporting claimants' health: A role for the personal adviser?

This study centres on the way in which welfare claimants' health-related needs are understood and addressed within the new welfare-to-work landscape. The study takes a specific interest in the role of the Personal Adviser, a central frontline practice figure who has previously been extensively involved in implementing UK welfare-to-work policy. A qualitative methodology underpinned by ethnographic principles was implemented. The study design aimed to take into consideration the macro, meso and micro-level factors that characterise the policy arena, provider organisations that provide employment support and frontline practice. The methods selected were: a documentary review, participant observation of the policy arena, observation of the practice arena and semi-structured interviews. The study found that the Personal Adviser is often at the heart of employment support delivery. Personal Advisers are expected to be competent in adopting different roles, some of which might conflict and cause tensions, when meeting the diverse needs of claimants who have health conditions. These findings raise important questions about the legitimacy and preparedness of Personal Advisers' practice in relation to supporting claimants' health. The findings also found that Prime Work Programme provider organisations had proposed varied levels of health-related support provision, and some of their models had a lack of prominence to health. This raises concerns about equity, quality and adequacy of any support being provided. The need for welfare policy to retain a health focus has been shown to be crucial, and integration between the NHS and employment provision needs to be improved, especially at the frontline. This study has contributed new knowledge about the nature of health-related support within the policy and practice context, and the Personal Adviser's role in supporting claimants with health conditions in the newly emerging world of welfare-to-work

Undergraduate Student Catalog 2020-2021

The central pillars of Qatar University’s mission are highlighted through this document, namely the provision of high-quality education and the pursuit of an active role in the development of Qatari society. The courses described here have been designed, reviewed and assessed to meet the highest educational standards, with a strong focus on the knowledge and skill-based learning that is needed for a graduate to be competitive in today’s labor market and in graduate education pursuits. The many of the academic programs have attained independent external accreditation from internationally recognized associations, to cater to the needs of the country’s ambitious development course

Health inequalities in Scotland and England : the translation of ideas between research and policy

EThOS - Electronic Theses Online ServiceGBUnited Kingdo

Federal Register

Daily publication of the U.S. Office of the Federal Register contains rules and regulations, proposed legislation and rule changes, and other notices, including "Presidential proclamations and Executive Orders, Federal agency documents having general applicability and legal effect, documents required to be published by act of Congress, and other Federal agency documents of public interest" (p. ii). Table of Contents starts on page iii

Baseline description of the Benthic Biotopes for two Long-Term Ecological Research (LTER) stations in Algoa Bay, Agulhas ecoregion, South Africa

Shallow coastal marine ecosystems provide important resources to society but are increasingly under threat from anthropogenic impacts. These systems form an interface between land and sea, providing valuable resources. Global environmental change, overexploitation, habitat transformation, pollution and policies aimed at short-term socio-economic gains are driving the loss of natural resources, productivity and biodiversity. Consequently, a comprehensive and holistic understanding of the current and future states of marine ecosystems is essential. This requires insight into the processes involved in maintaining genetic, species, habitat, community and biotope diversity at an ecosystem level. An understanding of ecosystem processes and the ability to detect changes in biodiversity, biotopes, seascape composition and ecosystem functioning require observation made over time and space. In response to this need, Long-Term Ecological Research (LTER) programmes such as those established by the South African Environmental Observation Network (SAEON) supported by the Department of Science and Innovation’s (DSI) Shallow Marine and Coastal Research Infrastructure (SMCRI) collect long–term observational data from different environments and systems. This study aimed to evaluate data collected as part of the developmental phase of a newly established benthic research platform. The datasets constructed are used to develop a baseline description of the benthic biotopes of two study areas within Algoa Bay. It aimed to gain insight into the feasibility of the methods used and the value of the data and derived essential biological variables (EVB). Assessing the sustainability of the programme over time was done through a practical evaluation of the methodology to be used and the technical feasibility of data collection and analysis. It furthermore aimed to assess the data usefulness in describing biodiversity at various scales and its sensitivity in reporting change. This pilot study provides valuable insight into data collection methodologies and introduces new sampling platforms. The baseline dataset consisted of data collected during the first 18 months of the SAEON Benthic Ecosystem Long-Term Ecological Research (BELTER) platform in Algoa Bay, Agulhas ecoregion, Western Indian Ocean, South Africa. This comprised of the collection of images for 150 m2 of the researched benthic seascape and 306 stereo video data streams. The 77 760 point dataset allowed the identification of 12 substrate types, 7 biotopes, 44 sub-biotope units, 377 sessile and sedentary benthic species and 51 ichthyofauna species. The described habitat and benthic communities and the defined benthic biotopes allowed for the assessment of biotope heterogeneity and the construction of a provisional distribution map for the broader biotopes. This work includes a study into the infra- and supra-benthic ichthyofauna associated with the defined biotopes and investigates the role habitat and benthic communities play in the distribution of these fish 3 assemblages. Lastly, it assesses the value of morphological traits and diversity indices for describing and comparing abiotic and biotic components of observed systems within the shallow coastal marine seascape. This study shows that species composition differs significantly between biotopes with habitat type playing a key role in the composition of the benthos. Substrate type, consolidated or unconsolidated, depth and the composition of the soft sediment is the most important determining factors. The White Sands Reef station has a higher species diversity than the St Croix Island Complex station with a higher percentage cover associated with the hard substrate. The dissimilarity between biotopes and communities are generally high although similarity within the biotopes or communities was found to be relatively low. This was considered indicative of high heterogeneity within the biotopes and a patch or mosaic-like distribution of communities within the broader biotope. A fine-scale a posteriori analysis of the data collected confirmed the high heterogeneous nature of both habitat and communities within the broader biotope. The description of the abiotic and biotic variables resulted in the identification of a diverse suite of biotope subunits. The character of the biotope hinges not only on the composition of the substrate and biota present but the contribution of smaller distinct biotopes subunits, their distribution and representation within broader biotopes and the degree these are shared with other broader biotopes. The distribution of these biotope units at different scales is believed to be important in understanding inherent diversity, niche partitioning and connectivity within a highly heterogeneous seascape. Ichthyofauna associated with the broader biotopes were indicative of the substrate type. Low profile reef systems with interspaced sandy stretches supported both reef fish and those typically associated with sandy substrates. Benthic biotopes associated characteristically with higher profile reef systems and less sand or soft sediment were mainly utilized by reef-associated fish species. Substrate type, depth and seasonality were found to be important factors in the observed composition and distribution of ichthyofauna over the seascape. Although fish species were found to have a wide distribution and made use of multiple biotopes the average abundance of the species within the observed assemblages differed. Analysis of ichthyofauna species composition indicated that observed fish assemblages were homogeneous within five of the seven biotopes. Broader biotopes that were found to be significantly different between sample locations are characterised by a diverse complement of biotope subunits and are highly heterogeneous. Traits and diversity indices are important tools for assessing and comparing different systems within the seascape, both spatially and temporally. The classification of the biota into broader phylogenetic groups indicated a significant difference between biotopes. This is especially useful when detailed 4 analysis or species identification is not possible or the skill set is not available. Morphological traits included in this study informed on the physical structure of the communities present and in combination with substrate type provided insight into the three-dimensional structure of the biotope. Species diversity, abundance, density estimates and the Shannon-Weiner diversity index were found to be the most useful diversity indices characterising and comparing biotopes. This was less so for ichthyofauna. Significant differences in the number of species observed were evident only between consolidated and unconsolidated dominated substrates. Although there was no significant difference in the number of individuals observed, both the Shannon-Weiner and Simpson Diversity indices were able to highlight differences in the fish assemblages observed for the different biotopes. The data collected, although permitting a comprehensive baseline assessment of the benthic environment for two research stations within the SAEON Algoa Bay LTER Sentinel Site, is temporally limited. The ichthyofauna dataset used was small and it is understood that the addition of length-frequency analysis of observed ichthyofauna will benefit our understanding of the biotope use by infra- and supra-benthic fish species over their life history within the larger seascape. Seasonal differences were evident and it is expected that datasets spanning several years, including LTER stations within different marine ecosystems types, will provide valuable insights on system dynamics in the short and long term both spatially and temporally. This study is the first attempt to evaluate the methodology developed and data collected in the South African Environmental Observation Network’s, Elwandle Coastal Node as part of the Shallow Marine and Coastal Research Infrastructure Benthic Ecosystem Long-Term Ecological Research (BELTER) platform. Newly designed and developed sample equipment and a sampling regime allowed for the collection of data on a long-term basis. The study was successful in the description of the biotope and biotope subunits for two research stations in Algoa Bay. It permitted the construction of comprehensive species lists for both benthic sessile and sedentary biota and the associated ichthyofauna. The subset of data used was successful in reporting on both spatial and temporal change. This work demonstrates that in the absence of detailed species identifications, traits may be used to describe habitat and community structure and report on abiotic and biotic biotope characteristics. This study furthermore allowed for the comparison of a comprehensive suite of diversity indices highlighting indices that may be especially useful in routine BELTER reporting.Thesis (PhD) -- Faculty of Science, Ichthyology and Fisheries Science, 202