Full Stack Optimization of Transformer Inference: a Survey

Recent advances in state-of-the-art DNN architecture design have been moving toward Transformer models. These models achieve superior accuracy across a wide range of applications. This trend has been consistent over the past several years since Transformer models were originally introduced. However, the amount of compute and bandwidth required for inference of recent Transformer models is growing at a significant rate, and this has made their deployment in latency-sensitive applications challenging. As such, there has been an increased focus on making Transformer models more efficient, with methods that range from changing the architecture design, all the way to developing dedicated domain-specific accelerators. In this work, we survey different approaches for efficient Transformer inference, including: (i) analysis and profiling of the bottlenecks in existing Transformer architectures and their similarities and differences with previous convolutional models; (ii) implications of Transformer architecture on hardware, including the impact of non-linear operations such as Layer Normalization, Softmax, and GELU, as well as linear operations, on hardware design; (iii) approaches for optimizing a fixed Transformer architecture; (iv) challenges in finding the right mapping and scheduling of operations for Transformer models; and (v) approaches for optimizing Transformer models by adapting the architecture using neural architecture search. Finally, we perform a case study by applying the surveyed optimizations on Gemmini, the open-source, full-stack DNN accelerator generator, and we show how each of these approaches can yield improvements, compared to previous benchmark results on Gemmini. Among other things, we find that a full-stack co-design approach with the aforementioned methods can result in up to 88.7x speedup with a minimal performance degradation for Transformer inference

Update on new aspects of the renin-angiotensin system in liver disease: clinical implications and new therapeutic options

A B S T R A C T The RAS (renin-angiotensin system) is now recognized as an important regulator of liver fibrosis and portal pressure. Liver injury stimulates the hepatic expression of components of the RAS, such as ACE (angiotensin-converting enzyme) and the AT 1 receptor [AngII (angiotensin II) type 1 receptor], which play an active role in promoting inflammation and deposition of extracellular matrix. In addition, the more recently recognized structural homologue of ACE, ACE2, is also up-regulated. ACE2 catalyses the conversion of AngII into Ang-(1-7) [angiotensin-(1-7)], and there is accumulating evidence that this 'alternative axis' of the RAS has anti-fibrotic, vasodilatory and anti-proliferative effects, thus counterbalancing the effects of AngII in the liver. The RAS is also emerging as an important contributor to the pathophysiology of portal hypertension in cirrhosis. Although the intrahepatic circulation in cirrhosis is hypercontractile in response to AngII, resulting in increased hepatic resistance, the splanchnic vasculature is hyporesponsive, promoting the development of the hyperdynamic circulation that characterizes portal hypertension. Both liver fibrosis and portal hypertension represent important therapeutic challenges for the clinician, and there is accumulating evidence that RAS blockade may be beneficial in these circumstances. The present review outlines new aspects of the RAS and explores its role in the pathogenesis and treatment of liver fibrosis and portal hypertension

Prolonged exposure of cortical neurons to oligomeric amyloid-β impairs NMDA receptor function via NADPH oxidase-mediated ROS production: protective effect of green tea (–)-epigallocatechin-3-gallate

Excessive production of Aβ (amyloid β-peptide) has been shown to play an important role in the pathogenesis of AD (Alzheimer's disease). Although not yet well understood, aggregation of Aβ is known to cause toxicity to neurons. Our recent study demonstrated the ability for oligomeric Aβ to stimulate the production of ROS (reactive oxygen species) in neurons through an NMDA (N-methyl-d-aspartate)-dependent pathway. However, whether prolonged exposure of neurons to aggregated Aβ is associated with impairment of NMDA receptor function has not been extensively investigated. In the present study, we show that prolonged exposure of primary cortical neurons to Aβ oligomers caused mitochondrial dysfunction, an attenuation of NMDA receptor-mediated Ca2+ influx and inhibition of NMDA-induced AA (arachidonic acid) release. Mitochondrial dysfunction and the decrease in NMDA receptor activity due to oligomeric Aβ are associated with an increase in ROS production. Gp91ds-tat, a specific peptide inhibitor of NADPH oxidase, and Mn(III)-tetrakis(4-benzoic acid)-porphyrin chloride, an ROS scavenger, effectively abrogated Aβ-induced ROS production. Furthermore, Aβ-induced mitochondrial dysfunction, impairment of NMDA Ca2+ influx and ROS production were prevented by pre-treatment of neurons with EGCG [(−)-epigallocatechin-3-gallate], a major polyphenolic component of green tea. Taken together, these results support a role for NADPH oxidase-mediated ROS production in the cytotoxic effects of Aβ, and demonstrate the therapeutic potential of EGCG and other dietary polyphenols in delaying onset or retarding the progression of AD

Single Nucleotide Polymorphisms That Increase Expression of the Guanosine Triphosphatase RAC1 Are Associated With Ulcerative Colitis

BACKGROUND & AIMS: RAC1 is a GTPase that has an evolutionarily conserved role in coordinating immune defenses, from plants to mammals. Chronic inflammatory bowel diseases (IBD) are associated with dysregulation of immune defenses. We studied the role of RAC1 in IBD using human genetic and functional studies and animal models of colitis. METHODS: We used a candidate gene approach to HapMap-Tag single nucleotide polymorphisms (SNPs) in a discovery cohort; findings were confirmed in 2 additional cohorts. RAC1 mRNA expression was examined from peripheral blood cells of patients. Colitis was induced in mice with conditional disruption of Rac1 in phagocytes by administration of dextran sulphate sodium (DSS). RESULTS: We observed a genetic association between RAC1 with ulcerative colitis (UC) in a discovery cohort, 2 independent replication cohorts, and in combined analysis for the SNPs rs10951982 (Pcombined UC = 3.3 × 10–8, odds ratio [OR]=1.43 [1.26–1.63]) and rs4720672 (Pcombined UC=4.7 × 10–6, OR=1.36 [1.19–1.58]). Patients with IBD who had the rs10951982 risk allele had increased expression of RAC1, compared to those without this allele. Conditional disruption of Rac1 in macrophage and neutrophils of mice protected them against DSS-induced colitis. CONCLUSION: Studies of human tissue samples and knockout mice demonstrated a role for the GTPase RAC1 in the development of UC; increased expression of RAC1 was associated with susceptibility to colitis

In-hospital complications after invasive strategy for the management of Non STEMI: women fare as well as men

<p>Abstract</p> <p>Background</p> <p>To analyze the in-hospital complication rate in women suffering from non-ST elevation myocardial infarction treated with percutaneous coronary intervention (PCI) compared to men.</p> <p>Methods</p> <p>The files of 479 consecutive patients (133 women and 346 men) suffering from a Non STEMI (Non ST-segment elevation myocardial infarction) between the January 1^st2006 and March 21^st2009 were retrospectively analyzed with special attention to every single complication occurring during hospital stay. Data were analyzed using nonparametric tests and are reported as median unless otherwise specified. A p value < .05 was considered significant.</p> <p>Results</p> <p>As compared to men, women were significantly older (75.8 <it>vs</it>. 65.2 years; p < .005). All cardiovascular risk factors but tobacco and hypertension were similar between the groups: men were noticeably more often smoker (p < .0001) and women more hypertensive (p < .005). No difference was noticed for pre-hospital cardiovascular drug treatment. However women were slightly more severe at entry (more Killip class IV; p = .0023; higher GRACE score for in-hospital death - p = .008 and CRUSADE score for bleeding - p < .0001). All the patients underwent PCI of the infarct-related artery after 24 or 48 hrs post admission without sex-related difference either for timing of PCI or primary success rate. During hospitalization, 130 complications were recorded. Though the event rate was slightly higher in women (30% <it>vs</it>. 26% - p = NS), no single event was significantly gender related. The logistic regression identified age and CRP concentration as the only predictive variables in the whole group. After splitting for genders, these parameters were still predictive of events in men. In women however, CRP was the only one with a borderline p value.</p> <p>Conclusions</p> <p>Our study does not support any gender difference for in-hospital adverse events in patients treated invasively for an acute coronary syndrome without ST-segment elevation and elevated troponin.</p

The contribution of refractoriness to arrhythmic substrate in hypokalemic Langendorff-perfused murine hearts

The clinical effects of hypokalemia including action potential prolongation and arrhythmogenicity suppressible by lidocaine were reproduced in hypokalemic (3.0 mM K(+)) Langendorff-perfused murine hearts before and after exposure to lidocaine (10 μM). Novel limiting criteria for local and transmural, epicardial, and endocardial re-excitation involving action potential duration (at 90% repolarization, APD(90)), ventricular effective refractory period (VERP), and transmural conduction time (Δlatency), where appropriate, were applied to normokalemic (5.2 mM K(+)) and hypokalemic hearts. Hypokalemia increased epicardial APD(90) from 46.6 ± 1.2 to 53.1 ± 0.7 ms yet decreased epicardial VERP from 41 ± 4 to 29 ± 1 ms, left endocardial APD(90) unchanged (58.2 ± 3.7 to 56.9 ± 4.0 ms) yet decreased endocardial VERP from 48 ± 4 to 29 ± 2 ms, and left Δlatency unchanged (1.6 ± 1.4 to 1.1 ± 1.1 ms; eight normokalemic and five hypokalemic hearts). These findings precisely matched computational predictions based on previous reports of altered ion channel gating and membrane hyperpolarization. Hypokalemia thus shifted all re-excitation criteria in the positive direction. In contrast, hypokalemia spared epicardial APD(90) (54.8 ± 2.7 to 60.6 ± 2.7 ms), epicardial VERP (84 ± 5 to 81 ± 7 ms), endocardial APD(90) (56.6 ± 4.2 to 63.7 ± 6.4 ms), endocardial VERP (80 ± 2 to 84 ± 4 ms), and Δlatency (12.5 ± 6.2 to 7.6 ± 3.4 ms; five hearts in each case) in lidocaine-treated hearts. Exposure to lidocaine thus consistently shifted all re-excitation criteria in the negative direction, again precisely agreeing with the arrhythmogenic findings. In contrast, established analyses invoking transmural dispersion of repolarization failed to account for any of these findings. We thus establish novel, more general, criteria predictive of arrhythmogenicity that may be particularly useful where APD(90) might diverge sharply from VERP

Antimicrobial resistance among migrants in Europe: a systematic review and meta-analysis

BACKGROUND: Rates of antimicrobial resistance (AMR) are rising globally and there is concern that increased migration is contributing to the burden of antibiotic resistance in Europe. However, the effect of migration on the burden of AMR in Europe has not yet been comprehensively examined. Therefore, we did a systematic review and meta-analysis to identify and synthesise data for AMR carriage or infection in migrants to Europe to examine differences in patterns of AMR across migrant groups and in different settings. METHODS: For this systematic review and meta-analysis, we searched MEDLINE, Embase, PubMed, and Scopus with no language restrictions from Jan 1, 2000, to Jan 18, 2017, for primary data from observational studies reporting antibacterial resistance in common bacterial pathogens among migrants to 21 European Union-15 and European Economic Area countries. To be eligible for inclusion, studies had to report data on carriage or infection with laboratory-confirmed antibiotic-resistant organisms in migrant populations. We extracted data from eligible studies and assessed quality using piloted, standardised forms. We did not examine drug resistance in tuberculosis and excluded articles solely reporting on this parameter. We also excluded articles in which migrant status was determined by ethnicity, country of birth of participants' parents, or was not defined, and articles in which data were not disaggregated by migrant status. Outcomes were carriage of or infection with antibiotic-resistant organisms. We used random-effects models to calculate the pooled prevalence of each outcome. The study protocol is registered with PROSPERO, number CRD42016043681. FINDINGS: We identified 2274 articles, of which 23 observational studies reporting on antibiotic resistance in 2319 migrants were included. The pooled prevalence of any AMR carriage or AMR infection in migrants was 25·4% (95% CI 19·1-31·8; I2 =98%), including meticillin-resistant Staphylococcus aureus (7·8%, 4·8-10·7; I2 =92%) and antibiotic-resistant Gram-negative bacteria (27·2%, 17·6-36·8; I2 =94%). The pooled prevalence of any AMR carriage or infection was higher in refugees and asylum seekers (33·0%, 18·3-47·6; I2 =98%) than in other migrant groups (6·6%, 1·8-11·3; I2 =92%). The pooled prevalence of antibiotic-resistant organisms was slightly higher in high-migrant community settings (33·1%, 11·1-55·1; I2 =96%) than in migrants in hospitals (24·3%, 16·1-32·6; I2 =98%). We did not find evidence of high rates of transmission of AMR from migrant to host populations. INTERPRETATION: Migrants are exposed to conditions favouring the emergence of drug resistance during transit and in host countries in Europe. Increased antibiotic resistance among refugees and asylum seekers and in high-migrant community settings (such as refugee camps and detention facilities) highlights the need for improved living conditions, access to health care, and initiatives to facilitate detection of and appropriate high-quality treatment for antibiotic-resistant infections during transit and in host countries. Protocols for the prevention and control of infection and for antibiotic surveillance need to be integrated in all aspects of health care, which should be accessible for all migrant groups, and should target determinants of AMR before, during, and after migration. FUNDING: UK National Institute for Health Research Imperial Biomedical Research Centre, Imperial College Healthcare Charity, the Wellcome Trust, and UK National Institute for Health Research Health Protection Research Unit in Healthcare-associated Infections and Antimictobial Resistance at Imperial College London

Oxygen tension regulates the miRNA profile and bioactivity of exosomes released from extravillous trophoblast cells - liquid biopsies for monitoring complications of pregnancy

Our understanding of how cells communicate has undergone a paradigm shift since the recent recognition of the role of exosomes in intercellular signaling. In this study, we investigated whether oxygen tension alters the exosome release and miRNA profile from extravillous trophoblast (EVT) cells, modifying their bioactivity on endothelial cells (EC). Furthermore, we have established the exosomal miRNA profile at early gestation in women who develop pre-eclampsia (PE) and spontaneous preterm birth (SPTB). HTR-8/SVneo cells were used as an EVT model. The effect of oxygen tension (i.e. 8% and 1% oxygen) on exosome release was quantified using nanocrystals (Qdot®) coupled to CD63 by fluorescence NTA. A real-time, live-cell imaging system (Incucyte™) was used to establish the effect of exosomes on EC. Plasma samples were obtained at early gestation (<18 weeks) and classified according to pregnancy outcomes. An Illumina TrueSeq Small RNA kit was used to construct a small RNA library from exosomal RNA obtained from EVT and plasma samples. The number of exosomes was significantly higher in EVT cultured under 1% compared to 8% oxygen. In total, 741 miRNA were identified in exosomes from EVT. Bioinformatic analysis revealed that these miRNA were associated with cell migration and cytokine production. Interestingly, exosomes isolated from EVT cultured at 8% oxygen increased EC migration, whilst exosomes cultured at 1% oxygen decreased EC migration. These changes were inversely proportional to TNF-α released from EC. Finally, we have identified a set of unique miRNAs in exosomes from EVT cultured at 1% oxygen and exosomes isolated from the circulation of mothers at early gestation, who later developed PE and SPTB. We suggest that aberrant exosomal signalling by placental cells is a common aetiological factor in pregnancy complications characterised by incomplete SpA remodeling and is therefore a clinically relevant biomarker of pregnancy complications.Grace Truong, Dominic Guanzon, Vyjayanthi Kinhal, Omar Elfeky, Andrew Lai, Sherri Longo, Zarin Nuzhat, Carlos Palma, Katherin Scholz-Romero, Ramkumar Menon, Ben W. Mol, Gregory E. Rice, Carlos Salomo

Non-equivalence of Wnt and R-spondin ligands during Lgr5+ intestinal stem-cell self-renewal

The canonical Wnt/β-catenin signaling pathway governs diverse developmental, homeostatic and pathologic processes. Palmitoylated Wnt ligands engage cell surface Frizzled (Fzd) receptors and Lrp5/6 co-receptors enabling β-catenin nuclear translocation and Tcf/Lef-dependent gene transactivation1–3. Mutations in Wnt downstream signaling components have revealed diverse functions presumptively attributed to Wnt ligands themselves, although direct attribution remains elusive, as complicated by redundancy between 19 mammalian Wnts and 10 Fzds1 and Wnt hydrophobicity2,3. For example, individual Wnt ligand mutations have not revealed homeostatic phenotypes in the intestinal epithelium4, an archetypal canonical Wnt pathway-dependent rapidly self-renewing tissue whose regeneration is fueled by proliferative crypt Lgr5+ intestinal stem cells (ISCs)5–9. R-spondin ligands (Rspo1–4) engage distinct Lgr4-6 and Rnf43/Znrf3 receptor classes10–13, markedly potentiate canonical Wnt/β-catenin signaling and induce intestinal organoid growth in vitro and Lgr5+ ISCs in vivo8,14–17. However, the interchangeability, functional cooperation and relative contributions of Wnt versus Rspo ligands to in vivo canonical Wnt signaling and ISC biology remain unknown. Here, we deconstructed functional roles of Wnt versus Rspo ligands in the intestinal crypt stem cell niche. We demonstrate that the default fate of Lgr5+ ISCs is lineage commitment, escape from which requires both Rspo and Wnt ligands. However, gain-of-function studies using Rspo versus a novel non-lipidated Wnt analog reveal qualitatively distinct, non-interchangeable roles for these ligands in ISCs. Wnts are insufficient to induce Lgr5+ ISC self-renewal, but rather confer a basal competency by maintaining Rspo receptor expression that enables Rspo to actively drive and specify the extent of stem cell expansion. This functionally non-equivalent yet cooperative interplay between Wnt and Rspo ligands establishes a molecular precedent for regulation of mammalian stem cells by distinct priming and self-renewal factors, with broad implications for precision control of tissue regeneration

Plastid phylogenomics resolves ambiguous relationships within the orchid family and provides a solid timeframe for biogeography and macroevolution

Recent phylogenomic analyses based on the maternally inherited plastid organelle have enlightened evolutionary relationships between the subfamilies of Orchidaceae and most of the tribes. However, uncertainty remains within several subtribes and genera for which phylogenetic relationships have not ever been tested in a phylogenomic context. To address these knowledge-gaps, we here provide the most extensively sampled analysis of the orchid family to date, based on 78 plastid coding genes representing 264 species, 117 genera, 18 tribes and 28 subtribes. Divergence times are also provided as inferred from strict and relaxed molecular clocks and birth–death tree models. Our taxon sampling includes 51 newly sequenced plastid genomes produced by a genome skimming approach. We focus our sampling efforts on previously unplaced clades within tribes Cymbidieae and Epidendreae. Our results confirmed phylogenetic relationships in Orchidaceae as recovered in previous studies, most of which were recovered with maximum support (209 of the 262 tree branches). We provide for the first time a clear phylogenetic placement for Codonorchideae within subfamily Orchidoideae, and Podochilieae and Collabieae within subfamily Epidendroideae. We also identify relationships that have been persistently problematic across multiple studies, regardless of the different details of sampling and genomic datasets used for phylogenetic reconstructions. Our study provides an expanded, robust temporal phylogenomic framework of the Orchidaceae that paves the way for biogeographical and macroevolutionary studies.Universidad de Costa Rica/[814-B8-257]/UCR/Costa RicaUniversidad de Costa Rica/[814-B6-140]/UCR/Costa RicaIDEA WILD/[]//Estados UnidosSociedad Colombiana de Orquideología/[]/SCO/ColombiaFundação de Amparo à Pesquisa do Estado de São Paulo/[11/08308-9]/FAPESP/BrasilFundação de Amparo à Pesquisa do Estado de São Paulo/[13/19124-1]/FAPESP/BrasilSwiss Orchid Foundation/[]//SuizaRoyal Botanic Gardens, Kew/[]//InglaterraSwedish Research Council/[2019-05191]//SueciaSwedish Foundation for Strategic Research/[FFL15-0196]/SSF/SueciaUCR::Vicerrectoría de Investigación::Unidades de Investigación::Ciencias Agroalimentarias::Jardín Botánico Lankester (JBL