Automating Systematic Literature Reviews with Natural Language Processing and Text Mining: a Systematic Literature Review

Objectives: An SLR is presented focusing on text mining based automation of SLR creation. The present review identifies the objectives of the automation studies and the aspects of those steps that were automated. In so doing, the various ML techniques used, challenges, limitations and scope of further research are explained. Methods: Accessible published literature studies that primarily focus on automation of study selection, study quality assessment, data extraction and data synthesis portions of SLR. Twenty-nine studies were analyzed. Results: This review identifies the objectives of the automation studies, steps within the study selection, study quality assessment, data extraction and data synthesis portions that were automated, the various ML techniques used, challenges, limitations and scope of further research. Discussion: We describe uses of NLP/TM techniques to support increased automation of systematic literature reviews. This area has attracted increase attention in the last decade due to significant gaps in the applicability of TM to automate steps in the SLR process. There are significant gaps in the application of TM and related automation techniques in the areas of data extraction, monitoring, quality assessment and data synthesis. There is thus a need for continued progress in this area, and this is expected to ultimately significantly facilitate the construction of systematic literature reviews

Towards automation of systematic reviews using natural language processing, machine learning, and deep learning: a comprehensive review.

Systematic reviews (SRs) constitute a critical foundation for evidence-based decision-making and policy formulation across various disciplines, particularly in healthcare and beyond. However, the inherently rigorous and structured nature of the SR process renders it laborious for human reviewers. Moreover, the exponential growth in daily published literature exacerbates the challenge, as SRs risk missing out on incorporating recent studies that could potentially influence research outcomes. This pressing need to streamline and enhance the efficiency of SRs has prompted significant interest in leveraging Artificial Intelligence (AI) techniques to automate various stages of the SR process. This review paper provides a comprehensive overview of the current AI methods employed for SR automation, a subject area that has not been exhaustively covered in previous literature. Through an extensive analysis of 52 related works and an original online survey, the primary AI techniques and their applications in automating key SR stages, such as search, screening, data extraction, and risk of bias assessment, are identified. The survey results offer practical insights into the current practices, experiences, opinions, and expectations of SR practitioners and researchers regarding future SR automation. Synthesis of the literature review and survey findings highlights gaps and challenges in the current landscape of SR automation using AI techniques. Based on these insights, potential future directions are discussed. This review aims to equip researchers and practitioners with a foundational understanding of the basic concepts, primary methodologies and recent advancements in AI-driven SR automation, while guiding computer scientists in exploring novel techniques to further invigorate and advance this field

Findings from a literature review

Mentzingen, H., António, N., & Bação, F. (2023). Automation of legal precedents retrieval: Findings from a literature review. International Journal of Intelligent Systems, 2023, 1-22. [6660983]. https://doi.org/10.21203/rs.3.rs-2292464/v1, https://doi.org/10.21203/rs.3.rs-2292464/v2, https://doi.org/10.1155/2023/6660983---This work was supported by national funds through FCT (Fundação para a Ciência e a Tecnologia), under the project-UIDB/04152/2020-Centro de Investigação em Gestão de Informação (MagIC)/NOVA IMS.Judges frequently rely their reasoning on precedents. Courts must preserve uniformity in decisions while, depending on the legal system, previous cases compel rulings. The search for methods to accurately identify similar previous cases is not new and has been a vital input, for example, to case-based reasoning (CBR) methodologies. This literature review offers a comprehensive analysis of the advancements in automating the identification of legal precedents, primarily focusing on the paradigm shift from manual knowledge engineering to the incorporation of Artificial Intelligence (AI) technologies such as natural language processing (NLP) and machine learning (ML). While multiple approaches harnessing NLP and ML show promise, none has emerged as definitively superior, and further validation through statistically significant samples and expert-provided ground truth is imperative. Additionally, this review employs text-mining techniques to streamline the survey process, providing an accurate and holistic view of the current research landscape. By delineating extant research gaps and suggesting avenues for future exploration, this review serves as both a summation and a call for more targeted, empirical investigations.publishersversionpublishe

Systematic literature review (SLR) automation: a systematic literature review

Context: A systematic literature review(SLR) is a methodology used to find and aggregate all relevant studies about a specific research question or topic of interest. Most of the SLR processes are manually conducted. Automating these processes can reduce the workload and time consumed by human. Method: we use SLR as a methodology to survey the literature about the technologies used to automate SLR processes. Result: from the collected data we found many work done to automate the study selection process but there is no evidence about automation of the planning and reporting process. Most of the authors use machine learning classifiers to automate the study selection process. From our survey, there are processes that are similar to the SLR process for which there are automatic techniques to perform them. Conclusion: Because of these results, we concluded that there should be more research done on the planning, reporting, data extraction and synthesizing processes of SLR

Using Informatics to Improve Autism Screening in a Pediatric Primary Care Practice

Background: According to the most recent report from the CDC (2018), autism spectrum disorder (ASD) affects approximately one in 59 children in the United States (U.S.). In 2007, the American Academy of Pediatrics (AAP) issued a strong recommendation for all primary care providers to screen children for autism, using a validated tool, at the 18 and 24-month well-child visits, in order to begin the referral process for more formal testing, and intervention, promptly. Despite the strong stance of the AAP and evidence supporting the importance of early intervention for children with ASD, not all primary care providers are screening for ASD or developmental delay. Purpose: To improve the percentage of eligible children, presenting for 18 and 24 month wellchild visits in a pediatric primary care office, who are screened for ASD, by integrating the Modified Checklist for Autism in Toddlers (M-CHAT) screening tool into the electronic medical record with tablets. The specific aims were to increase the percentage of children screened and improve the documentation of the screens performed. Methods: This quality improvement project utilized a before-after quantitative design to support the improvement. Reports were obtained for three months prior to the implementation of the tablets and process change, and again for three months following the implementation. Manual chart reviews were also performed to verify the data from the reports. The definition used for complete screening for this project included 1) presence of the completed screen in the medical record, 2) provider documentation of the result, interpretation, and plan if indicated, and 3) CPT code entry for charge capture completed in the electronic medical record. Results: The results of the project revealed improvements in overall percentages of eligible children screened for autism at D-H Nashua Pediatrics. The percentage of complete screening increased from 64.7% to 73.9% following the implementation of the project, a change which is statistically significant (t=31.6105, df=16,p=0.05). Each individual element was also tracked and those results showed that 1) the completeness of provider documentation related to the screening increased from 93.6% to 96% (t=41.3321, df=16, p=0.05) and 2) the M-CHAT screen was present in the electronic health record (EHR) 98.9% of the time, which was an increase from 84.6% (t=295.4084, df=16, p=0.05). The charge capture completion rate remained statistically unchanged at 76.5% (t=0.4664, df=16, p=0.05). Additionally, only one screening was noted to be missed altogether, out of 280 eligible children. Prior to the project, there were four missed screenings (out of 156 eligible children) captured by the chart reviews conducted over three months prior to the implementation of the project. Overall, the results show that the project resulted in an increase the percentage of M-CHAT screening, an increase in the presence of source documentation in the electronic health record (EHR), and more complete provider documentation related to the screening

Reporting Statistical Validity and Model Complexity in Machine Learning based Computational Studies

Background:: Statistical validity and model complexity are both important concepts to enhanced understanding and correctness assessment of computational models. However, information about these are often missing from publications applying machine learning. Aim: The aim of this study is to show the importance of providing details that can indicate statistical validity and complexity of models in publications. This is explored in the context of citation screening automation using machine learning techniques. Method: We built 15 Support Vector Machine (SVM) models, each developed using word2vec (average word) features --- and data for 15 review topics from the Drug Evaluation Review Program (DERP) of the Agency for Healthcare Research and Quality (AHRQ). Results: The word2vec features were found to be sufficiently linearly separable by the SVM and consequently we used the linear kernels. In 11 of the 15 models, the negative (majority) class used over 80% of its training data as support vectors (SVs) and approximately 45% of the positive training data. Conclusions: In this context, exploring the SVs revealed that the models are overly complex against ideal expectations of not more than 2%-5% (and preferably much less) of the training vectors

Artificial intelligence to automate the systematic review of scientific literature

Artificial intelligence (AI) has acquired notorious relevance in modern computing as it effectively solves complex tasks traditionally done by humans. AI provides methods to represent and infer knowledge, efficiently manipulate texts and learn from vast amount of data. These characteristics are applicable in many activities that human find laborious or repetitive, as is the case of the analysis of scientific literature. Manually preparing and writing a systematic literature review (SLR) takes considerable time and effort, since it requires planning a strategy, conducting the literature search and analysis, and reporting the findings. Depending on the area under study, the number of papers retrieved can be of hundreds or thousands, meaning that filtering those relevant ones and extracting the key information becomes a costly and error-prone process. However, some of the involved tasks are repetitive and, therefore, subject to automation by means of AI. In this paper, we present a survey of AI techniques proposed in the last 15 years to help researchers conduct systematic analyses of scientific literature. We describe the tasks currently supported, the types of algorithms applied, and available tools proposed in 34 primary studies. This survey also provides a historical perspective of the evolution of the field and the role that humans can play in an increasingly automated SLR process.Comment: 25 pages, 3 figures, 1 table, journal pape