1,696 research outputs found

    Systematic and fast scientific literature review for policy purposes :An exploration of using ASReview software about the effectiveness of policy instruments promoting sustainable agriculture

    Get PDF
    In the context of Article 3.1 of the compatibility law 2016 (Comptabiliteitswet) (CW 2016), which aims to bring about more scientific underpinning of policy, the House of Representatives in 2019 examined the extent to which policy proposals address the efficiency and effectiveness of the policy instruments to be deployed (Sneller &amp; Snels, 2019). They concluded that while they often address how these instruments should contribute to the objective, they often do not address to what extent.Making such statements about efficiency and effectiveness requires an understanding of scientific evidence. As a result, the Netherlands Environmental Assessment Agency (PBL) anticipates that they will receive more frequent questions from the government about what science says about the effectiveness of proposed policies. PBL already has a strong scientific orientation in its work and recognizes the great importance of a request for scientific foundation. But is also foresees that answering this type of question about scientific evidence for policy proposals can be difficult in practice. This is due to the short period in which these questions must generally be answered on the one hand and the time required for (systematic) literature review on the other. Conducting a (systematic) literature search using artificial intelligence (AI) could offer a solution to this problem and that is what this report is about.When using AI, the literature search is supported by a so-called learning algorithm, which learns as the process progresses to better and better assess what is relevant literature for the researcher. However, little experience has been gained with the use of such software in policy-oriented research. PBL therefore asked the UG to investigate how the open-source AI software ASReview could offer a solution in efficiently meeting the demand for scientific insights from the government. At PBL's request, the UG researched three substantive questions around the effectiveness of policy instruments for sustainable agriculture and used these to test the process of the AI supported literature review. In this research, the UG worked with a combination of AI-supported literature screening and sounding boards of academics who fed the search at the beginning and interpreted the results substantively at the end. Such a sounding board enables the distillation of substantive lessons in a relatively short time.The findings are: 1. It is possible to conduct a quick and good systematic literature search by combining AI-supported literature search with a sounding board of scientific experts. The search for scientific insights that could potentially provide substantiation yielded a database of 40.000 potentially relevant papers, from which a diverse set of 100 relevant papers were selected using AI. Using the sounding board of experts, an even smaller set of 12 papers was created from this list that were deemed most urgent for policymakers to study.2. Because the AI software ASReview puts content first and hides reputation of authors and journals during screening, objectivity and breadth is stimulated;3. However, ASReview does present other challenges that, if not taken into account, can compromise the objectivity of research in other ways;i) One of the main risks of using ASReview is what we call ‘trap formation’, especially when there is a short time frame for screening. This means that you end up on a particular 'track' of articles on a particular (sub)topic, which means that other also relevant articles are not found. The broader the query and the less time, the more this poses a risk to the efficiency and reliability of the screening. By using certain settings for the screening, this can be taken into account to a certain extent.ii) The amount of efficiency gains that can be achieved with ASReview depends on the breadth and multidisciplinarity of the research question. The topic of sustainable agriculture has many facets, both in terms of instruments and outcomes. This makes it more difficult for the program to quickly learn what is most relevant, so screening will take more time;iii) AI is not a fully automated process. Use of the program requires skill from the researcher to drive the algorithm and expert knowledge to start the process and expert knowledge to interpret the results.4. The content of this research has resulted in two sets of scientific articles: a Top 100 and a Top 12.5. The first end result of the Systematic Review is a list (Top 100) of relevant articles ranked by their "scientific recognition." Here, because of the requirement for speed in the process, scientific recognition is simply operationalized as a combination of the number of citations of the article per year and the impact factor of the journal (as a measure of the seriousness of the blind review process) in which they were published. While there are caveats to this method of ranking, it does provide a way to make a large amount of knowledge manageable within a relatively short period of time, and in doing so, gives policymakers and researchers a foothold that they can study "the most important first''. 6. The second end result is a selection from the top 100 by the scientific experts: which ones are most important for policy? The six multi-disciplinary scholars each selected three papers that they felt were most important for policymakers and researchers to take to task: combined, this yielded a set of 12 articles. The Top 12 is a manageable set of articles that was selected quickly, and can be studied in content in a short period of time while being largely systematic.7. In conclusion. What does the process tested here offer compared to what we might call "the standard quick search for scientific evidence" of a PBL staff member? This search often consists of manually consulting Google Scholar and/or individually contacting a scientific expert. Compared to manual searches via Scholar, searching the literature with ASReview offers the opportunity to systematically review a vast amount of studies for relevance. After each selection by the researcher, the entire database is reordered. Ultimately, it has been shown, this leads to a greater diversity of studies than a manual search process. Compared to contacting experts individually, the added value of utilizing a group of experts in this study is not only their diversity of expertise, but also that they all reflect on the same scientific dataset and choose from it (quickly and rationally) the most relevant ones for policy. This is a much more systematic process than asking for their scientific views separately.<br/

    A study of early indication citation metrics

    No full text
    Research outputs are growing in number and frequency, assisted by a greater number of publication mediums and platforms via which material can be disseminated. At the same time, the requirement to find acceptable, timely, objective measurements of research "quality" has become more important. Historically, citations have been used as an independent indication of the significance of scholarly material. However, citations are very slow to accrue since they can only be made by subsequently published material. This enforces a delay of a number of years before the citation impact of a publication can be accurately judged. By contrast, each new citation establishes a large number of co-citation relationships between that publication and older material whose citation impact is already well established. By taking advantage of this co-citation property, this thesis investigates the possibility of developing a metric that can provide an earlier indicator of a publication's citation impact. This thesis proposes a new family of cocitation based impact measures, describes a system to evaluate their effectiveness against a large citation database, and justifies the results of this evaluation against an analysis of a diverse range of research metric

    The Janus Faced Scholar:a Festschrift in honour of Peter Ingwersen

    Get PDF

    Making Institutional Repositories Work

    Get PDF
    Making Institutional Repositories Work takes novices as well as seasoned practitioners through the practical and conceptual steps necessary to develop a functioning institutional repository, customized to the needs and culture of the home institution. The first section covers all aspects of system platforms, including hosted and open-source options, big data capabilities and integration, and issues related to discoverability. The second section addresses policy issues, from the basics to open-source and deposit mandates. The third section focuses on recruiting and even creating content. Authors in this section will address the ways that different disciplines tend to have different motivations for deposit, as well as the various ways that institutional repositories can serve as publishing platforms. The fourth section covers assessment and success measures for all involved—librarians, deans, and administrators. The theory and practice of traditional metrics, alt metrics, and peer review receive chapter-length treatment. The fifth section provides case studies that include a boots-on-the-ground perspective of issues raised in the first four sections. By noting trends and potentialities, this final section, authored by Executive Director of SPARC Heather Joseph, makes future predictions and helps managers position institutional repositories to be responsive to change and even shape the evolution of scholarly communication.https://docs.lib.purdue.edu/purduepress_ebooks/1040/thumbnail.jp

    Writing Rangers

    Get PDF

    Towards an understanding of talent management as a phenomenon-driven field using bibliometric and content analysis

    Get PDF
    This review adopts a phenomenon-driven approach in reviewing the talent management (TM) literature, applying methods derived from bibliometrics and content analysis to evaluate the state of the field and derive implications for research and practice unbiased towards a-priori assumptions of which frameworks or methods are most adequate. Based on analyses of publication volume, journals and their impact factors, most cited articles and authors, preferred methods, and represented countries, we assess whether TM should be approached as an embryonic, growth, or mature phenomenon, and examine dominant (i.e., resource-based view, international human resource management, employee assessment, and institutionalism) versus ‘alternative’ (i.e., knowledge management, career management, strength-based approach, and social exchange theory) theoretical frameworks. Our goal is to assist TM researchers in positioning their work more explicitly vis-à-vis current debates in the existing literature and encourage them to think about which approach best fits their research aims, questions, and designsPeer ReviewedPostprint (author’s final draft

    A systematic review of whole class, subject based, pedagogies with reported outcomes for the academic and social inclusion of pupils with special educational needs in mainstream classrooms

    Get PDF
    Schools across the world have responded to international and national initiatives designed to further the development of inclusive education. In England, there is a statutory requirement for all schools to provide effective learning opportunities for all pupils (QCA, 2000) and children with special educational needs (SEN) are positioned as having a right to be within mainstream classrooms accessing an appropriate curriculum (SENDA, 2001). Previous reviews which have sought to identify classroom practices that support the inclusion of children with SEN have been technically non-systematic and hence a need for a systematic review within this area has been identified (Nind et al., 2004; Rix et al., 2006). This systematic literature review is the last in a series of three
    • …
    corecore