Data extraction methods for systematic review (semi)automation: A living systematic review [version 1; peer review: awaiting peer review]

Background: The reliable and usable (semi)automation of data extraction can support the field of systematic review by reducing the workload required to gather information about the conduct and results of the included studies. This living systematic review examines published approaches for data extraction from reports of clinical studies. Methods: We systematically and continually search MEDLINE, Institute of Electrical and Electronics Engineers (IEEE), arXiv, and the dblp computer science bibliography databases. Full text screening and data extraction are conducted within an open-source living systematic review application created for the purpose of this review. This iteration of the living review includes publications up to a cut-off date of 22 April 2020. Results: In total, 53 publications are included in this version of our review. Of these, 41 (77%) of the publications addressed extraction of data from abstracts, while 14 (26%) used full texts. A total of 48 (90%) publications developed and evaluated classifiers that used randomised controlled trials as the main target texts. Over 30 entities were extracted, with PICOs (population, intervention, comparator, outcome) being the most frequently extracted. A description of their datasets was provided by 49 publications (94%), but only seven (13%) made the data publicly available. Code was made available by 10 (19%) publications, and five (9%) implemented publicly available tools. Conclusions: This living systematic review presents an overview of (semi)automated data-extraction literature of interest to different types of systematic review. We identified a broad evidence base of publications describing data extraction for interventional reviews and a small number of publications extracting epidemiological or diagnostic accuracy data. The lack of publicly available gold-standard data for evaluation, and lack of application thereof, makes it difficult to draw conclusions on which is the best-performing system for each data extraction target. With this living review we aim to review the literature continually

Data extraction methods for systematic review (semi)automation: Update of a living systematic review [version 2; peer review: 3 approved]

Background: The reliable and usable (semi)automation of data extraction can support the field of systematic review by reducing the workload required to gather information about the conduct and results of the included studies. This living systematic review examines published approaches for data extraction from reports of clinical studies. Methods: We systematically and continually search PubMed, ACL Anthology, arXiv, OpenAlex via EPPI-Reviewer, and the dblp computer science bibliography. Full text screening and data extraction are conducted within an open-source living systematic review application created for the purpose of this review. This living review update includes publications up to December 2022 and OpenAlex content up to March 2023. Results: 76 publications are included in this review. Of these, 64 (84%) of the publications addressed extraction of data from abstracts, while 19 (25%) used full texts. A total of 71 (93%) publications developed classifiers for randomised controlled trials. Over 30 entities were extracted, with PICOs (population, intervention, comparator, outcome) being the most frequently extracted. Data are available from 25 (33%), and code from 30 (39%) publications. Six (8%) implemented publicly available tools Conclusions: This living systematic review presents an overview of (semi)automated data-extraction literature of interest to different types of literature review. We identified a broad evidence base of publications describing data extraction for interventional reviews and a small number of publications extracting epidemiological or diagnostic accuracy data. Between review updates, trends for sharing data and code increased strongly: in the base-review, data and code were available for 13 and 19% respectively, these numbers increased to 78 and 87% within the 23 new publications. Compared with the base-review, we observed another research trend, away from straightforward data extraction and towards additionally extracting relations between entities or automatic text summarisation. With this living review we aim to review the literature continually

Effect of Federal Policy Changes on International Students Pursuing Higher Education Studies in the United States

Background: In 2018, President Donald J. Trump announced that his administration would place restrictions on international students seeking to pursue higher education degrees in the United States. American institutions of higher education protested these policy changes, because international students represent a significant social and cultural contribution to their system and provide a source of revenue. The restrictions on international students were not overwhelming, primarily consisting of increased visa fees and threats stating that misbehavior in the country would result in immediate deportation. Although these demands do not typically deter international students, some individuals view these restrictions as part of an overall trend of anti-immigrant sentiment in the United States. The goal of this study was to investigate the impact of these new restrictions on the education of international students in the United States. Methods: The population, intervention, comparison, and outcome (PICO) question format was used to formulate the research question, centered on international students seeking to complete their higher education in the United States. The databases used for this study were ProQuest, JSTOR, LexisNexis, and Google Scholar. Results: The movement to place restrictions on international students in the United States is a recent development, and no statistically significant effects can presently be determined. Government funding for public universities, who market their programs to international students, has been reduced. Conclusions: This research demonstrates that international student attendance at American universities was declining before the immigration restrictions were implemented. Based on current data, it is too early to determine how immigration restrictions will impact American universities, and more time will be needed to evaluate the impact of President Trump’s policies

Appraising the Potential Uses and Harms of LLMs for Medical Systematic Reviews

Medical systematic reviews play a vital role in healthcare decision making and policy. However, their production is time-consuming, limiting the availability of high-quality and up-to-date evidence summaries. Recent advancements in large language models (LLMs) offer the potential to automatically generate literature reviews on demand, addressing this issue. However, LLMs sometimes generate inaccurate (and potentially misleading) texts by hallucination or omission. In healthcare, this can make LLMs unusable at best and dangerous at worst. We conducted 16 interviews with international systematic review experts to characterize the perceived utility and risks of LLMs in the specific context of medical evidence reviews. Experts indicated that LLMs can assist in the writing process by drafting summaries, generating templates, distilling information, and crosschecking information. They also raised concerns regarding confidently composed but inaccurate LLM outputs and other potential downstream harms, including decreased accountability and proliferation of low-quality reviews. Informed by this qualitative analysis, we identify criteria for rigorous evaluation of biomedical LLMs aligned with domain expert views.Comment: 18 pages, 2 figures, 8 tables. Accepted as an EMNLP 2023 main pape

Assessment of contextualised representations in detecting outcome phrases in clinical trials

Automating the recognition of outcomes reported in clinical trials using machine learning has a huge potential of speeding up access to evidence necessary in healthcare decision-making. Prior research has however acknowledged inadequate training corpora as a challenge for the Outcome detection (OD) task. Additionally, several contextualized representations like BERT and ELMO have achieved unparalleled success in detecting various diseases, genes, proteins, and chemicals, however, the same cannot be emphatically stated for outcomes, because these models have been relatively under-tested and studied for the OD task. We introduce "EBM-COMET", a dataset in which 300 PubMed abstracts are expertly annotated for clinical outcomes. Unlike prior related datasets that use arbitrary outcome classifications, we use labels from a taxonomy recently published to standardize outcome classifications. To extract outcomes, we fine-tune a variety of pre-trained contextualized representations, additionally, we use frozen contextualized and context-independent representations in our custom neural model augmented with clinically informed Part-Of-Speech embeddings and a cost-sensitive loss function. We adopt strict evaluation for the trained models by rewarding them for correctly identifying full outcome phrases rather than words within the entities i.e. given an outcome "systolic blood pressure", the models are rewarded a classification score only when they predict all 3 words in sequence, otherwise, they are not rewarded. We observe our best model (BioBERT) achieve 81.5\% F1, 81.3\% sensitivity and 98.0\% specificity. We reach a consensus on which contextualized representations are best suited for detecting outcomes from clinical-trial abstracts. Furthermore, our best model outperforms scores published on the original EBM-NLP dataset leader-board scores

Data extraction methods for systematic review (semi)automation: Update of a living systematic review [version 2; peer review: 3 approved]

Background: The reliable and usable (semi)automation of data extraction can support the field of systematic review by reducing the workload required to gather information about the conduct and results of the included studies. This living systematic review examines published approaches for data extraction from reports of clinical studies. Methods: We systematically and continually search PubMed, ACL Anthology, arXiv, OpenAlex via EPPI-Reviewer, and the dblp computer science bibliography. Full text screening and data extraction are conducted within an open-source living systematic review application created for the purpose of this review. This living review update includes publications up to December 2022 and OpenAlex content up to March 2023. Results: 76 publications are included in this review. Of these, 64 (84%) of the publications addressed extraction of data from abstracts, while 19 (25%) used full texts. A total of 71 (93%) publications developed classifiers for randomised controlled trials. Over 30 entities were extracted, with PICOs (population, intervention, comparator, outcome) being the most frequently extracted. Data are available from 25 (33%), and code from 30 (39%) publications. Six (8%) implemented publicly available tools Conclusions: This living systematic review presents an overview of (semi)automated data-extraction literature of interest to different types of literature review. We identified a broad evidence base of publications describing data extraction for interventional reviews and a small number of publications extracting epidemiological or diagnostic accuracy data. Between review updates, trends for sharing data and code increased strongly: in the base-review, data and code were available for 13 and 19% respectively, these numbers increased to 78 and 87% within the 23 new publications. Compared with the base-review, we observed another research trend, away from straightforward data extraction and towards additionally extracting relations between entities or automatic text summarisation. With this living review we aim to review the literature continually

Sentence Classification with Hierarchical Neural Networks for Rhetorical Sections Extraction

Υπόβαθρο: Εκατομμύρια επιστημονικά άρθρα και επιστημονικές εργασίες δημοσιεύονται κάθε χρόνο, καθιστώντας την έρευνα για σχετική βιβλιογραφία όλο και πιο δύσκολη με κάθε μέρα που περνά. Ως εκ τούτου, οι σαφείς και ενημερωτικές περιλήψεις έχουν καταστεί απαραίτητο μέσο για να εντοπίζουν οι ερευνητές τις επιθυμητές πληροφορίες εγκαίρως και με αποτελεσματικό τρόπο. Πολλές περιλήψεις, ωστόσο, εξακολουθούν να στερούνται κοινών ρητορικών δομικών στοιχείων τα οποία θα βελτίωναν τους επικοινωνιακούς τους σκοπούς στο πλαίσιο του ακαδημαϊκού λόγου. Στόχος: Στην παρούσα διατριβή στοχεύουμε να εξετάσουμε την αποτελεσματικότητα των μοντέλων ταξινόμησης προτάσεων για την εξαγωγή ρητορικών ενοτήτων σε περιλήψεις διαφορετικών τομέων και δομών και να δημιουργήσουμε ένα εργαλείο που υτοματοποιεί αυτήν τη διαδικασία. Μέθοδος: Τα μοντέλα ταξινόμησης προτάσεων που χρησιμοποιήθηκαν εδώ βασίστηκαν σε ένα ιεραρχικό νευρωνικό δίκτυο (HNN) που έχει εκπαιδευτεί σε τρία διαφορετικά σύνολα δεδομένων. Αποτέλεσμα: Τα αποτελέσματά μας δείχνουν ότι τα μοντέλα μας επιβεβαιώνουν την ”state of the art” απόδοσή τους (SOTA) σε περιλήψεις του ίδιου επιστημονικού πεδίου με εκείνες που εκπαιδεύτηκαν, αλλά η διαπεδιακή ακρίβειά τους μειώνεται σημαντικά ειδικά όταν εφαρμόζονται σε μη κλασσικά δομημένες περιλήψεις. Συμπέρασμα: Ένα ακριβές εργαλείο για την απόκτηση των ρητορικών τμημάτων των περιλήψεων μπορεί να αποτελέσει τη βάση για ένα μεγαλύτερο σύστημα που θα μπορεί να συνοψίζει τις πληροφορίες, βοηθώντας έτσι σε μεγάλο βαθμό την επιτάχυνση της διαδικασίας της βιβλιογραφικής έρευνας.Background: Millions of scholarly articles and scientific papers are being published each year, making the search for relevant literature harder with each passing day. Clear and informative abstracts have therefore become an essential medium for researchers to locate their desired information in a timely and efficient manner. Many abstracts however, still lack common rhetorical structural elements that would improve their communicative purposes within the context of academic discourse. Objective: In the present thesis we aim to review the efficacy of sentence classification models for rhetorical sections extraction on abstracts of different domains and structures and create a tool that automates this process. Method: The sentence classification models used here were based on a hierarchical neural network (HNN) that has been trained on three different datasets. Result: Our results show that our models manage to confirm their state of the art (SOTA) performance on abstracts of the same scientific field with the ones they were trained in, but their inter­domain accuracy drops significantly especially when applied to unordinarily structured abstracts. Conclusion: An accurate tool for obtaining the rhetorical sections of abstracts can become the basis for a larger framework that could summarize information, helping tremendously to speed up the process of literature research

An NLP Analysis of Health Advice Giving in the Medical Research Literature

Health advice – clinical and policy recommendations – plays a vital role in guiding medical practices and public health policies. Whether or not authors should give health advice in medical research publications is a controversial issue. The proponents of actionable research advocate for the more efficient and effective transmission of science evidence into practice. The opponents are concerned about the quality of health advice in individual research papers, especially that in observational studies. Arguments both for and against giving advice in individual studies indicate a strong need for identifying and accessing health advice, for either practical use or quality evaluation purposes. However, current information services do not support the direct retrieval of health advice. Compared to other natural language processing (NLP) applications, health advice has not been computationally modeled as a language construct either. A new information service for directly accessing health advice should be able to reduce information barriers and to provide external assessment in science communication. This dissertation work built an annotated corpus of scientific claims that distinguishes health advice according to its occurrence and strength. The study developed NLP-based prediction models to identify health advice in the PubMed literature. Using the annotated corpus and prediction models, the study answered research questions regarding the practice of advice giving in medical research literature. To test and demonstrate the potential use of the prediction model, it was used to retrieve health advice regarding the use of hydroxychloroquine (HCQ) as a treatment for COVID-19 from LitCovid, a large COVID-19 research literature database curated by the National Institutes of Health. An evaluation of sentences extracted from both abstracts and discussions showed that BERT-based pre-trained language models performed well at detecting health advice. The health advice prediction model may be combined with existing health information service systems to provide more convenient navigation of a large volume of health literature. Findings from the study also show researchers are careful not to give advice solely in abstracts. They also tend to give weaker and non-specific advice in abstracts than in discussions. In addition, the study found that health advice has appeared consistently in the abstracts of observational studies over the past 25 years. In the sample, 41.2% of the studies offered health advice in their conclusions, which is lower than earlier estimations based on analyses of much smaller samples processed manually. In the abstracts of observational studies, journals with a lower impact are more likely to give health advice than those with a higher impact, suggesting the significance of the role of journals as gatekeepers of science communication. For the communities of natural language processing, information science, and public health, this work advances knowledge of the automated recognition of health advice in scientific literature. The corpus and code developed for the study have been made publicly available to facilitate future efforts in health advice retrieval and analysis. Furthermore, this study discusses the ways in which researchers give health advice in medical research articles, knowledge of which could be an essential step towards curbing potential exaggeration in the current global science communication. It also contributes to ongoing discussions of the integrity of scientific output. This study calls for caution in advice-giving in medical research literature, especially in abstracts alone. It also calls for open access to medical research publications, so that health researchers and practitioners can fully review the advice in scientific outputs and its implications. More evaluative strategies that can increase the overall quality of health advice in research articles are needed by journal editors and reviewers, given their gatekeeping role in science communication

Prevention and Treatment of Sarcopenia

Sarcopenia represents the decline in skeletal muscle mass and function with age, characterized by the muscle fiber's quality, strength, muscle endurance, and metabolic ability decreasing, as well as the fat and connective tissue growing.Reduction of muscle strength with aging leads to loss of functional capacity, causing disability, mortality, and other adverse health outcomes. Because of the increase of the proportion of elderly in the population, sarcopenia-related morbidity will become an increasing area of health care resource utilization.Diagnostic screening consists of individuation of body composition, assessed by DEXA, anthropometry, bioelectrical impedance, MRI, or CT scan. Management is possible with resistance training exercise and vibration therapy, nutritional supplements, and pharmacological treatment.The book includes articles from different nationalities, treating the experimental and medical applications of sarcopenia. The consequences of sarcopenia in frailty are treated in relation to other associated pathologies or lesions, as femoral neck fractures and hepatocellular carcinoma

Methodological challenges in the evidence synthesis of health outcomes of digital health technologies [védés előtt]

Medical devices and pharmaceuticals are worlds apart, but healthcare would be impossible without them. Digital biomarkers are the subject of this thesis defined as objective, measurable, physiological, and behavioural parameters collected using wearable, portable, implantable, or digestible digital devices. Since the 1970s, systematic reviews and meta-analyses have dominated medical evidence synthesis. They provide medical decision-making evidence. To avoid biases and maintain methodological quality, the Cochrane Handbook recommends systematic reviews follow certain procedures during study stages. This thesis comprises six hypotheses related to digital biomarkers. The first hypothesis aimed to evaluate the suitability of using tools provided by the World Health Organization (WHO), including ICD-11 (International Classification of Diseases, 11th Revision), ICHI (International Classification of Health Interventions), and ICF (International Classification of Functioning, Disability and Health), for categorizing populations, interventions, outcomes, and behavioral/physiological data in studies involving digital biomarkers. The results indicated that these tools were not applicable for categorizing digital biomarker studies as a whole. However, further analysis revealed that these tools were suitable for categorizing digital biomarker studies involving non-general populations or populations with specific diseases. The second hypothesis focused on comparing the statistical power of direct and indirect digital biomarkers. The results indicated that there was no significant difference in power between these two types of digital biomarkers (p-value > 0.05). The next three hypotheses compared the characteristics of systematic reviews and meta-analyses of digital biomarker-based interventions with those of non-digital biomarkers or pharmaceuticals. The comparisons were made in terms of methodological quality, quality of evidence, and publication bias. Although all these hypotheses revealed non-significant differences between the two groups (p-values > 0.05), the results showed that both digital biomarkers and non-digital biomarkers or pharmaceuticals systematic reviews did not exhibit high methodological quality or quality of evidence. The Medical Device Regulation (MDR) has significantly improved European medical device regulatory standards, addressing the above concerns and improving clinical evidence. Despite MDR implementation delays, digital health technology evidence requirements are rising. Companies that achieve these higher clinical requirements will survive and obtain access to large interconnected markets, while those that fail may lose their market authorisation. Thus, medical technology enterprises may gain a competitive edge by strategically planning and executing extensive clinical investigations to provide high-quality clinical data. Developing these essential skills needs immediate attention and effort. Digital health investors should actively monitor industry players' evidence quality and clinical trial competence, since these characteristics may significantly increase company risk