When Is Multilinguality a Curse? Language Modeling for 250 High- and Low-Resource Languages

Multilingual language models are widely used to extend NLP systems to low-resource languages. However, concrete evidence for the effects of multilinguality on language modeling performance in individual languages remains scarce. Here, we pre-train over 10,000 monolingual and multilingual language models for over 250 languages, including multiple language families that are under-studied in NLP. We assess how language modeling performance in each language varies as a function of (1) monolingual dataset size, (2) added multilingual dataset size, (3) linguistic similarity of the added languages, and (4) model size (up to 45M parameters). We find that in moderation, adding multilingual data improves low-resource language modeling performance, similar to increasing low-resource dataset sizes by up to 33%. Improvements depend on the syntactic similarity of the added multilingual data, with marginal additional effects of vocabulary overlap. However, high-resource languages consistently perform worse in multilingual pre-training scenarios. As dataset sizes increase, adding multilingual data begins to hurt performance for both low-resource and high-resource languages, likely due to limited model capacity (the "curse of multilinguality"). These results suggest that massively multilingual pre-training may not be optimal for any languages involved, but that more targeted models can significantly improve performance

Transforming growth factor-β1 and diabetic nephropathy

Transforming growth factor-β1 (TGF-β1) is established to be involved in the pathogenesis of diabetic nephropathy. The diabetic milieu enhances oxidative stress and induces the expression of TGF-β1. TGF-β1 promotes cell hypertrophy and extracellular matrix accumulation in the mesangium, which decreases glomerular filtration rate and leads to chronic renal failure. Recently, TGF-β1 has been demonstrated to regulate urinary albumin excretion by both increasing glomerular permeability and decreasing reabsorption in the proximal tubules. TGF-β1 also increases urinary excretion of water, electrolytes and glucose by suppressing tubular reabsorption in both normal and diabetic conditions. Although TGF-β1 exerts hypertrophic and fibrogenic effects in diabetic nephropathy, whether suppression of the function of TGF-β1 can be an option to prevent or treat the complication is still controversial. This is partly because adrenal production of mineralocorticoids could be augmented by the suppression of TGF-β1. However, differentiating the molecular mechanisms for glomerulosclerosis from those for the suppression of the effects of mineralocorticoids by TGF-β1 may assist in developing novel therapeutic strategies for diabetic nephropathy. In this review, we discuss recent findings on the role of TGF-β1 in diabetic nephropathy

Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models

Abstract grammatical knowledge - of parts of speech and grammatical patterns - is key to the capacity for linguistic generalization in humans. But how abstract is grammatical knowledge in large language models? In the human literature, compelling evidence for grammatical abstraction comes from structural priming. A sentence that shares the same grammatical structure as a preceding sentence is processed and produced more readily. Because confounds exist when using stimuli in a single language, evidence of abstraction is even more compelling from crosslingual structural priming, where use of a syntactic structure in one language primes an analogous structure in another language. We measure crosslingual structural priming in large language models, comparing model behavior to human experimental results from eight crosslingual experiments covering six languages, and four monolingual structural priming experiments in three non-English languages. We find evidence for abstract monolingual and crosslingual grammatical representations in the models that function similarly to those found in humans. These results demonstrate that grammatical representations in multilingual language models are not only similar across languages, but they can causally influence text produced in different languages.Comment: Accepted at EMNLP 202

Addressing privilege and oppression in counselor training and practice: A qualitative analysis

This qualitative study explored how 16 counselors conceptualize and address issues of privilege and oppression in the counseling session as well as how they perceive their training with respect to these constructs. In an effort to bridge multicultural training and counselor practice, implications for counselor training are provided based on the clinical and academic experiences counselors reported. Additionally, future research directions are included

Estrada de Ferro Caravelas: trilhos pioneiros na trajetória socioeconômica do sul do Espírito Santo

O objetivo desta dissertação de História Social das Relações Políticas é discutir a relevância da Estrada de Ferro Caravelas para o desenvolvimento socioeconômico do Sul do Espírito Santo. Apresentamos uma breve abordagem histórica dos caminhos do ferro na Europa e no Brasil do século XIX. Em seguida, buscaremos compreender a conjuntura da economia nacional e a relação da economia cafeeira com a implantação das ferrovias no cenário nacional e capixaba. Apresentaremos também as dificuldades da implantação das ferrovias no cenário estadual e analisaremos os motivos do primeiro trecho ferroviário ter sido construído na região sul do Espírito Santo. Finalmente, evidenciaremos a importância da Estrada no cenário de desenvolvimento econômico da região, sob uma abordagem que englobe os aspectos econômicos e sociais; o impacto da implantação da Estrada de Ferro Caravelas e sua anexação à Leopoldina Railway Company ampliando o trecho ferroviário que ligava Cachoeiro de Itapemirim a Alegre e, posteriormente, a Espera Feliz. Utilizamos como principais fontes documentos oficiais e periódicos da época, com a perspectiva metodológica que buscou analisar os discursos presentes nessas fontes

Systemic inflammation, coagulopathy, and acute renal insufficiency following endovascular thoracoabdominal aortic aneurysm repair

ObjectiveTo characterize the inflammatory and coagulopathic response after endovascular thoracoabdominal aortic aneurysm (TAAA) repair and to evaluate the effect of the response on postoperative renal function.MethodsFrom July 2005 to June 2008, 42 patients underwent elective endovascular repair of a TAAA using custom designed multi-branched stent-grafts at a single academic institution. Four patients were excluded from the analysis. White blood cell count (WBC), platelet count, prothrombin time (PT), and creatinine were measured in all patients. In the last nine patients, interleukin-6 (IL-6), protein C, Factor V, d-dimers, cystatin C, and neutrophil gelatinase-associated lipocalin (NGAL) levels were also measured. Change in lab values were expressed as a percentage of baseline values.ResultsThe 30-day mortality rate was 5% (2/38). All patients (n = 38) had a higher WBC (mean ± SD: 139 ± 80%, P < .0001), lower platelet count (56 ± 15%, P < .0001), and higher PT (median: 17%, Interquartile range (IQR) 12%-22%, P < .0001) after stent-graft insertion. Twelve of 38 patients (32%) developed postoperative acute renal insufficiency (>50% rise in creatinine). Patients with renal insufficiency had significantly larger changes in WBC (178 ± 100% vs 121 ± 64%, P = .04) and platelet count (64 ± 17% vs 52 ± 12%, P = .02) compared with those without renal insufficiency. All patients (n = 9) had significant increases in NGAL (182 ± 115%, P = .008) after stent-graft insertion. Six of nine patients (67%) had increased cystatin C (35 ± 43%, P = .04) after stent-graft insertion, with a greater rise in those with postoperative renal insufficiency (87 ± 32% vs 8 ± 13%, P = .02). IL-6 levels were markedly increased in all patients (n = 9) after repair (9840 ± 6160%, P = .008). Protein C (35 ± 10%, P = .008) and Factor V levels (28 ± 20%, P = .008) were uniformly decreased, while d-dimers were elevated after repair in all patients (310 ± 213%, P = .008).ConclusionsLeukocytosis and thrombocytopenia were uniform following endovascular TAAA repair, and the severity of the response correlated with post-operative renal dysfunction. Elevation of a sensitive marker of renal injury (NGAL) suggests that renal injury may occur in all patients after stent-graft insertion

Outcomes after redo aortobifemoral bypass for aortoiliac occlusive disease

ObjectivePatients presenting with occluded aortobifemoral (ABF) bypass grafts are managed with a variety of techniques. Redo ABF (rABF) bypass procedures are infrequently performed because of concerns about procedural complexity and morbidity. The purpose of this analysis was to compare midterm results of rABF bypass with those of primary ABF (pABF) bypass for aortoiliac occlusive disease to determine if there are significant differences in outcomes.MethodsA retrospective review was performed of all patients undergoing ABF bypass for occlusive disease between January 2002 and March 2012. A total of 19 patients underwent rABF bypass and 194 received pABF bypass during that period. Data for an indication- and comorbidity-matched case-control cohort of 19 elective pABF bypass patients were collected for comparison to the rABF bypass group. Primary end points included rate of major complications as well as 30-day and all-cause mortality. Secondary end points were amputation-free survival and freedom from major adverse limb events.ResultsThe rABF bypass patients more frequently underwent prior extra-anatomic or lower extremity bypass operations compared with pABF bypass patients (P = .02); however, no difference was found in the incidence of prior failed endovascular iliac intervention (P = .4). By design, indications for the rABF and pABF bypass groups were the same (claudication, n = 6/6 [31.6%]; P = 1; critical limb ischemia, n = 13/13 [78.4%]; P = 1). Aortic access was more frequently by retroperitoneal exposure in the rABF bypass group (n = 13 vs n = 1; P < .0001), and a significantly higher proportion of the rABF bypass patients required concomitant infrainguinal bypass or intraprocedural adjuncts such as profundaplasty (n = 14 vs n = 5; P = .01). The rABF bypass patients experienced greater blood loss (1097 ± 983 mL vs 580 ± 457 mL; P = .02), received more intraoperative fluids (3400 ± 1422 mL vs 2279 ± 993 mL; P = .01), and had longer overall procedure times (408 ± 102 minutes vs 270 ± 48 minutes; P < .0001). Length of stay (days ± standard deviation) was similar (pABF bypass, 11.2 ± 10.4; rABF bypass, 9.1 ± 4.5; P = .7), and no 30-day or in-hospital deaths occurred in either group. Similar rates of major complications occurred in the two groups (pABF bypass, n = 6 [31.6%]; rABF bypass, n = 4 [21.1%]; observed difference, 9.5%; 95% confidence interval, −17.6% to 36.7%; P = .7). Two-year freedom from major adverse limb events (±standard error mean) was 82% ± 9% vs 78% ± 10% for pABF and rABF bypass patients (log-rank, P = .6). Two-year amputation-free survival was 90 ± 9% vs 89 ± 8% between pABF and rABF bypass patients (P = .5). Two-year survival was 91% ± 9% and 90% ± 9% for pABF and rABF bypass patients (P = .8).ConclusionsPatients undergoing rABF bypass have higher procedural complexity compared with pABF bypass as evidenced by greater operative time, blood loss, and need for adjunctive procedures. However, similar perioperative morbidity, mortality, and midterm survival occurred in comparison to pABF bypass patients. These results support a role for rABF bypass in selected patients

Prediction of graft patency and mortality after distal revascularization and interval ligation for hemodialysis access-related hand ischemia

ObjectiveThe treatment goals of access-related hand ischemia (ARHI) are to reverse symptoms and salvage the access. Many procedures have been described, but the optimal treatment strategy remains unresolved. In an effort to guide clinical decision making, this study was undertaken to document our outcomes for distal revascularization and interval ligation (DRIL) and to identify predictors of bypass patency and patient mortality.MethodsA retrospective review was performed of all patients who underwent DRIL at the University of Florida from 2002 to 2011. Diagnosis of ARHI was based primarily upon clinical symptoms with noninvasive studies used to corroborate in equivocal cases. Patient demographics, procedure-outcome variables, and reinterventions were recorded. Bypass patency and mortality were estimated using cumulative incidence and Kaplan-Meier methodology, respectively. Cumulative incidence and Cox regression analysis were performed to determine predictors of bypass patency and mortality, respectively.ResultsA total of 134 DRILs were performed in 126 patients (mean [standard deviation] age, 57 [12] years) following brachial artery-based access. The postoperative complication rate was 27% (19% wound), and 30-day mortality was 2%. The wrist-brachial index and digital brachial index increased 0.31 (0.25) and 0.25 (0.29), respectively. Symptoms resolved in 82% of patients, and 85% continued to use their access. Cumulative incidences (± standard error of the mean) of loss of primary and primary-assisted patency rates were 5% ± 2% and 4% ± 2% at 1 year and 22% ± 5% and 18% ± 5% at 5 years, respectively, with mean follow-up of 14.8 months. Univariate predictors of primary patency failure were DRIL complications (odds ratio [OR], 3.3; 95% confidence interval [CI], 1.2-8.9; P = .02), configuration other than brachiobasilic/brachiocephalic autogenous access (OR, 3.4; 95% CI, 1.4-8.3; P = .009), and two or more prior access attempts (OR, 4.1; 95% CI, 1.6-10.4; P = .004). Brachiocephalic access configuration (OR, 0.2; 95% CI, 0.04-0.8; P = .02) and autogenous vein conduit (OR, 0.2; 95% CI, 0.06-0.58; P = .004) were predictors of improved bypass patency. All-cause mortality was 28% and 79% at 1 and 5 years, respectively. Multivariable predictors of mortality were age >40 (hazard ratio [HR], 8.3; 95% CI, 2.5-33.3; P = .0004), grade 3 ischemia (HR, 2.6; 95% CI, 1.5-4.6; P = .0008), complication from DRIL (HR, 2.4; 95% CI, 1.3-4.5; P = .004), and smoking history (HR, 2.2; 95% CI, 1.3-4; P = .007). Patients with no prior access attempts had lower predicted mortality (HR, 0.5; 95% CI, 0.3-0.9; P = .02).ConclusionsThe DRIL procedure effectively improves distal perfusion and reverses the symptoms of ARHI while salvaging the access, but the long-term survival of these patients is poor. Given the poor survival, preoperative risk stratification is critical. Patients at high risk for DRIL failure and mortality may be best served with alternate remedial procedures