On the Impact of Cross-Domain Data on German Language Models

Traditionally, large language models have been either trained on general web crawls or domain-specific data. However, recent successes of generative large language models, have shed light on the benefits of cross-domain datasets. To examine the significance of prioritizing data diversity over quality, we present a German dataset comprising texts from five domains, along with another dataset aimed at containing high-quality data. Through training a series of models ranging between 122M and 750M parameters on both datasets, we conduct a comprehensive benchmark on multiple downstream tasks. Our findings demonstrate that the models trained on the cross-domain dataset outperform those trained on quality data alone, leading to improvements up to $4.45\%$ over the previous state-of-the-art. The models are available at https://huggingface.co/ikim-uk-essenComment: 13 pages, 1 figure, accepted at Findings of the Association for Computational Linguistics: EMNLP 202

Prognostic model to predict postoperative acute kidney injury in patients undergoing major gastrointestinal surgery based on a national prospective observational cohort study.

Background: Acute illness, existing co-morbidities and surgical stress response can all contribute to postoperative acute kidney injury (AKI) in patients undergoing major gastrointestinal surgery. The aim of this study was prospectively to develop a pragmatic prognostic model to stratify patients according to risk of developing AKI after major gastrointestinal surgery. Methods: This prospective multicentre cohort study included consecutive adults undergoing elective or emergency gastrointestinal resection, liver resection or stoma reversal in 2-week blocks over a continuous 3-month period. The primary outcome was the rate of AKI within 7 days of surgery. Bootstrap stability was used to select clinically plausible risk factors into the model. Internal model validation was carried out by bootstrap validation. Results: A total of 4544 patients were included across 173 centres in the UK and Ireland. The overall rate of AKI was 14·2 per cent (646 of 4544) and the 30-day mortality rate was 1·8 per cent (84 of 4544). Stage 1 AKI was significantly associated with 30-day mortality (unadjusted odds ratio 7·61, 95 per cent c.i. 4·49 to 12·90; P < 0·001), with increasing odds of death with each AKI stage. Six variables were selected for inclusion in the prognostic model: age, sex, ASA grade, preoperative estimated glomerular filtration rate, planned open surgery and preoperative use of either an angiotensin-converting enzyme inhibitor or an angiotensin receptor blocker. Internal validation demonstrated good model discrimination (c-statistic 0·65). Discussion: Following major gastrointestinal surgery, AKI occurred in one in seven patients. This preoperative prognostic model identified patients at high risk of postoperative AKI. Validation in an independent data set is required to ensure generalizability

Clinical features, socioeconomic status, management, short and long-term outcomes of patients with acute myocardial infarction: Phase I results of PEACE MENA registry.

BackgroundThe Program for the Evaluation and Management of Cardiac Events in the Middle East and North Africa (PEACE MENA) is a prospective registry program in Arabian countries that involves in patients with acute myocardial infarction (AMI) or acute heart failure (AHF).MethodsThis prospective, multi-center, multi-country study is the first report of the baseline characteristics and outcomes of inpatients with AMI who were enrolled during the first 14-month recruitment phase. We report the clinical characteristics, socioeconomic, educational levels, and management, in-hospital, one month and one-year outcomes.ResultsBetween April 2019 and June 2020, 1377 patients with AMI were enrolled (79.1% males) from 16 Arabian countries. The mean age (± SD) was 58 ± 12 years. Almost half of the population had a net income ConclusionIn the MENA region, patients with AMI present at a young age and have a high burden of cardiac risk factors. Most of the patients in the registry have a low income and low educational status. There is heterogeneity among key performance indicators of AMI management among various Arabian countries