A Trip Towards Fairness: Bias and De-Biasing in Large Language Models

An outbreak in the popularity of transformer-based Language Models (such as GPT (Brown et al., 2020) and PaLM (Chowdhery et al., 2022)) has opened the doors to new Machine Learning applications. In particular, in Natural Language Processing and how pre-training from large text, corpora is essential in achieving remarkable results in downstream tasks. However, these Language Models seem to have inherent biases toward certain demographics reflected in their training data. While research has attempted to mitigate this problem, existing methods either fail to remove bias altogether, degrade performance, or are expensive. This paper examines the bias produced by promising Language Models when varying parameters and pre-training data. Finally, we propose a de-biasing technique that produces robust de-bias models that maintain performance on downstream tasks

Outcome after Surgery for Iatrogenic Acute Type A Aortic Dissection

(1) Background: Acute Stanford type A aortic dissection (TAAD) may complicate the outcome of cardiovascular procedures. Data on the outcome after surgery for iatrogenic acute TAAD is scarce. (2) Methods: The European Registry of Type A Aortic Dissection (ERTAAD) is a multicenter, retrospective study including patients who underwent surgery for acute TAAD at 18 hospitals from eight European countries. The primary outcomes were in-hospital mortality and 5-year mortality. Twenty-seven secondary outcomes were evaluated. (3) Results: Out of 3902 consecutive patients who underwent surgery for acute TAAD, 103 (2.6%) had iatrogenic TAAD. Cardiac surgery (37.8%) and percutaneous coronary intervention (36.9%) were the most frequent causes leading to iatrogenic TAAD, followed by diagnostic coronary angiography (13.6%), transcatheter aortic valve replacement (10.7%) and peripheral endovascular procedure (1.0%). In hospital mortality was 20.5% after cardiac surgery, 31.6% after percutaneous coronary intervention, 42.9% after diagnostic coronary angiography, 45.5% after transcatheter aortic valve replacement and nihil after peripheral endovascular procedure (p = 0.092), with similar 5-year mortality between different subgroups of iatrogenic TAAD (p = 0.710). Among 102 propensity score matched pairs, in-hospital mortality was significantly higher among patients with iatrogenic TAAD (30.4% vs. 15.7%, p = 0.013) compared to those with spontaneous TAAD. This finding was likely related to higher risk of postoperative heart failure (35.3% vs. 10.8%, p < 0.0001) among iatrogenic TAAD patients. Five-year mortality was comparable between patients with iatrogenic and spontaneous TAAD (46.2% vs. 39.4%, p = 0.163). (4) Conclusions: Iatrogenic origin of acute TAAD is quite uncommon but carries a significantly increased risk of in-hospital mortality compared to spontaneous TAAD

Baseline risk factors of in-hospital mortality after surgery for acute type A aortic dissection: an ERTAAD study

Background Surgery for type A aortic dissection (TAAD) is associated with high risk of mortality. Current risk scoring methods have a limited predictive accuracy.Methods Subjects were patients who underwent surgery for acute TAAD at 18 European centers of cardiac surgery from the European Registry of Type A Aortic Dissection (ERTAAD).Results Out of 3,902 patients included in the ERTAAD, 2,477 fulfilled the inclusion criteria. In the validation dataset (2,229 patients), the rate of in-hospital mortality was 18.4%. The rate of composite outcome (in-hospital death, stroke/global ischemia, dialysis, and/or acute heart failure) was 41.2%, and 10-year mortality rate was 47.0%. Logistic regression identified the following patient-related variables associated with an increased risk of in-hospital mortality [area under the curve (AUC), 0.755, 95% confidence interval (CI), 0.729-0.780; Brier score 0.128]: age; estimated glomerular filtration rate; arterial lactate; iatrogenic dissection; left ventricular ejection fraction &lt;= 50%; invasive mechanical ventilation; cardiopulmonary resuscitation immediately before surgery; and cerebral, mesenteric, and peripheral malperfusion. The estimated risk score was associated with an increased risk of composite outcome (AUC, 0.689, 95% CI, 0.667-0.711) and of late mortality [hazard ratio (HR), 1.035, 95% CI, 1.031-1.038; Harrell's C 0.702; Somer's D 0.403]. In the validation dataset (248 patients), the in-hospital mortality rate was 16.1%, the composite outcome rate was 41.5%, and the 10-year mortality rate was 49.1%. The estimated risk score was predictive of in-hospital mortality (AUC, 0.703, 95% CI, 0.613-0.793; Brier score 0.121; slope 0.905) and of composite outcome (AUC, 0.682, 95% CI, 0.614-0.749). The estimated risk score was predictive of late mortality (HR, 1.035, 95% CI, 1.031-1.038; Harrell's C 0.702; Somer's D 0.403), also when hospital deaths were excluded from the analysis (HR, 1.024, 95% CI, 1.018-1.031; Harrell's C 0.630; Somer's D 0.261).Conclusions The present analysis identified several baseline clinical risk factors, along with preoperative estimated glomerular filtration rate and arterial lactate, which are predictive of in-hospital mortality and major postoperative adverse events after surgical repair of acute TAAD. These risk factors may be valuable components for risk adjustment in the evaluation of surgical and anesthesiological strategies aiming to improve the results of surgery for TAAD.Clinical Trial Registration https://clinicaltrials.gov, identifier NCT04831073

The AGILE Mission

AGILE is an Italian Space Agency mission dedicated to observing the gamma-ray Universe. The AGILE's very innovative instrumentation for the first time combines a gamma-ray imager (sensitive in the energy range 30 MeV-50 GeV), a hard X-ray imager (sensitive in the range 18-60 keV), a calorimeter (sensitive in the range 350 keV-100 MeV), and an anticoincidence system. AGILE was successfully launched on 2007 April 23 from the Indian base of Sriharikota and was inserted in an equatorial orbit with very low particle background. Aims. AGILE provides crucial data for the study of active galactic nuclei, gamma-ray bursts, pulsars, unidentified gamma-ray sources, galactic compact objects, supernova remnants, TeV sources, and fundamental physics by microsecond timing. Methods. An optimal sky angular positioning (reaching 0.1 degrees in gamma- rays and 1-2 arcmin in hard X-rays) and very large fields of view (2.5 sr and 1 sr, respectively) are obtained by the use of Silicon detectors integrated in a very compact instrument. Results. AGILE surveyed the gamma- ray sky and detected many Galactic and extragalactic sources during the first months of observations. Particular emphasis is given to multifrequency observation programs of extragalactic and galactic objects. Conclusions. AGILE is a successful high-energy gamma-ray mission that reached its nominal scientific performance. The AGILE Cycle-1 pointing program started on 2007 December 1, and is open to the international community through a Guest Observer Program

Outcome after Surgery for Iatrogenic Acute Type A Aortic Dissection

(1) Background: Acute Stanford type A aortic dissection (TAAD) may complicate the outcome of cardiovascular procedures. Data on the outcome after surgery for iatrogenic acute TAAD is scarce. (2) Methods: The European Registry of Type A Aortic Dissection (ERTAAD) is a multicenter, retrospective study including patients who underwent surgery for acute TAAD at 18 hospitals from eight European countries. The primary outcomes were in-hospital mortality and 5-year mortality. Twenty-seven secondary outcomes were evaluated. (3) Results: Out of 3902 consecutive patients who underwent surgery for acute TAAD, 103 (2.6%) had iatrogenic TAAD. Cardiac surgery (37.8%) and percutaneous coronary intervention (36.9%) were the most frequent causes leading to iatrogenic TAAD, followed by diagnostic coronary angiography (13.6%), transcatheter aortic valve replacement (10.7%) and peripheral endovascular procedure (1.0%). In hospital mortality was 20.5% after cardiac surgery, 31.6% after percutaneous coronary intervention, 42.9% after diagnostic coronary angiography, 45.5% after transcatheter aortic valve replacement and nihil after peripheral endovascular procedure (p = 0.092), with similar 5-year mortality between different subgroups of iatrogenic TAAD (p = 0.710). Among 102 propensity score matched pairs, in-hospital mortality was significantly higher among patients with iatrogenic TAAD (30.4% vs. 15.7%, p = 0.013) compared to those with spontaneous TAAD. This finding was likely related to higher risk of postoperative heart failure (35.3% vs. 10.8%, p &lt; 0.0001) among iatrogenic TAAD patients. Five-year mortality was comparable between patients with iatrogenic and spontaneous TAAD (46.2% vs. 39.4%, p = 0.163). (4) Conclusions: Iatrogenic origin of acute TAAD is quite uncommon but carries a significantly increased risk of in-hospital mortality compared to spontaneous TAAD

Pat-in-the-Loop: Declarative Knowledge for Controlling Neural Networks

The dazzling success of neural networks over natural language processing systems is imposing an urgent need to control their behavior with simpler, more direct declarative rules. In this paper, we propose Pat-in-the-Loop as a model to control a specific class of syntax-oriented neural networks by adding declarative rules. In Pat-in-the-Loop, distributed tree encoders allow to exploit parse trees in neural networks, heat parse trees visualize activation of parse trees, and parse subtrees are used as declarative rules in the neural network. Hence, Pat-in-the-Loop is a model to include human control in specific natural language processing (NLP)-neural network (NN) systems that exploit syntactic information, which we will generically call Pat. A pilot study on question classification showed that declarative rules representing human knowledge, injected by Pat, can be effectively used in these neural networks to ensure correctness, relevance, and cost-effective

A Simplified Approach for the Estimation of the Ventilatory Compensation Point

ONORATI, P., D. MARTOLINI, G. VALLI, P. LAVENEZIANA, P. MARINELLI, E. ANGELICI, and P. PALANGE. A Simplified Approach for the Estimation of the Ventilatory Compensation Point. Med. Sci. Sports Exerc., Vol. 44, No. 4, pp. 716-724, 2012. Incremental cardiopulmonary exercise test with gas exchange measurement is the gold standard for the identification of the ventilatory compensation point (VCP). It has previously been demonstrated that the change in the slope of increment of minute ventilation over HR (Delta V-E/Delta HR) can be used alternatively to the ventilatory equivalent for CO2 (Delta V-E/Delta CO2) method for detection of VCP in healthy subjects undergoing cycle ergometer (C) incremental exercise. The same evaluation during treadmill (T) incremental exercise and comparison between C and T have not yet been performed. Purpose: We analyzed, during both C and T incremental exercises, the V-E/HR and the respiratory rate (RR)/HR relationships, expressed either as slope or as an absolute value. We hypothesized that changes in the slope of increment of the two relationships could represent a reliable method for VCP detection, regardless of exercise mode and protocol. Methods: Fourteen healthy male subjects (age = 31 +/- 7 yr (mean +/- SD)) underwent two T incremental exercises-fast (F-T) and slow (S-T) protocols (8 km.h(-1), 2% (F-T) and 1% (S-T) grade per minute)-and one C incremental exercise (30 W.min(-1)). O-2 uptake (VO2), VCO2, V-E, HR, and RR were measured breath by breath. Results: A good between-method agreement in the detection of VCP by the Delta V-E/Delta CO2, Delta V-E/Delta HR, and the Delta RR/Delta HR slope changes was found in both T protocols and C. No differences (C vs T and F-T VS S-T) were found in the slope of the Delta V-E/Delta HR and Delta RR/Delta HR relationships after the VCP and in the Delta V-E/Delta HR and RR/HR absolute values at VCP. Conclusions: In healthy young males, the Delta V-E/Delta HR and Delta RR/Delta HR relationships during T and C incremental exercises can be reliably used to detect the VCP as an alternative to the ventilatory equivalent method

The Dark Side of the Language: Pre-trained Transformers in the DarkNet

Pre-trained Transformers are challenging human performances in many natural language processing tasks. The gigantic datasets used for pre-training seem to be the key for their success on existing tasks. In this paper, we explore how a range of pre-trained natural language understanding models perform on truly novel and unexplored data, provided by classification tasks over a DarkNet corpus. Surprisingly, results show that syntactic and lexical neural networks largely outperform pre-trained Transformers. This seems to suggest that pre-trained Transformers have serious difficulties in adapting to radically novel texts