EdgeTran: Co-designing Transformers for Efficient Inference on Mobile Edge Platforms

Automated design of efficient transformer models has recently attracted significant attention from industry and academia. However, most works only focus on certain metrics while searching for the best-performing transformer architecture. Furthermore, running traditional, complex, and large transformer models on low-compute edge platforms is a challenging problem. In this work, we propose a framework, called ProTran, to profile the hardware performance measures for a design space of transformer architectures and a diverse set of edge devices. We use this profiler in conjunction with the proposed co-design technique to obtain the best-performing models that have high accuracy on the given task and minimize latency, energy consumption, and peak power draw to enable edge deployment. We refer to our framework for co-optimizing accuracy and hardware performance measures as EdgeTran. It searches for the best transformer model and edge device pair. Finally, we propose GPTran, a multi-stage block-level grow-and-prune post-processing step that further improves accuracy in a hardware-aware manner. The obtained transformer model is 2.8$\times$ smaller and has a 0.8% higher GLUE score than the baseline (BERT-Base). Inference with it on the selected edge device enables 15.0% lower latency, 10.0$\times$ lower energy, and 10.8$\times$ lower peak power draw compared to an off-the-shelf GPU

TransCODE: Co-design of Transformers and Accelerators for Efficient Training and Inference

Automated co-design of machine learning models and evaluation hardware is critical for efficiently deploying such models at scale. Despite the state-of-the-art performance of transformer models, they are not yet ready for execution on resource-constrained hardware platforms. High memory requirements and low parallelizability of the transformer architecture exacerbate this problem. Recently-proposed accelerators attempt to optimize the throughput and energy consumption of transformer models. However, such works are either limited to a one-sided search of the model architecture or a restricted set of off-the-shelf devices. Furthermore, previous works only accelerate model inference and not training, which incurs substantially higher memory and compute resources, making the problem even more challenging. To address these limitations, this work proposes a dynamic training framework, called DynaProp, that speeds up the training process and reduces memory consumption. DynaProp is a low-overhead pruning method that prunes activations and gradients at runtime. To effectively execute this method on hardware for a diverse set of transformer architectures, we propose ELECTOR, a framework that simulates transformer inference and training on a design space of accelerators. We use this simulator in conjunction with the proposed co-design technique, called TransCODE, to obtain the best-performing models with high accuracy on the given task and minimize latency, energy consumption, and chip area. The obtained transformer-accelerator pair achieves 0.3% higher accuracy than the state-of-the-art pair while incurring 5.2$\times$ lower latency and 3.0$\times$ lower energy consumption

An Evolutionary Algorithm for Multi-criteria Resource Constrained Project Scheduling Problem based On PSO

AbstractThis paper is regarding the Particle Swarm Optimization (PSO)-based approach for the solution of the resource-constrained project scheduling problem with the purpose of minimizing cost. In order to evaluate the performance of the PSO based approach for the resource-constrained project scheduling problem, computational analyses are given. As per the results the application of PSO to project scheduling is achievable

Nuclear Matter in Intense Magnetic Field and Weak Processes

We study the effect of magnetic field on the dominant neutrino emission processes in neutron stars.The processes are first calculated for the case when the magnetic field does not exceed the critical value to confine electrons to the lowest Landau state.We then consider the more important case of intense magnetic field to evaluate the direct URCA and the neutronisation processes. In order to estimate the effect we derive the composition of cold nuclear matter at high densities and in beta equilibrium, a situation appropriate for neutron stars. The hadronic interactions are incorporated through the exchange of scalar and vector mesons in the frame work of relativistic mean field theory. In addition the effects of anomalous magnetic moments of nucleons are also considered.Comment: 29 pages (LaTeX) including 7 figure

O Mediterrâneo enquanto metáfora da mestiçagem: Novas leituras sobre o modelo europeu na América Latina dos anos 1920

After the First World War, we can observe in the Latin American society a strong transformation in the perception of the Europe as a civilization model. New movements in art and literature start to rethink the National Identities in Latin America and in the whole subcontinent born a criticism against the importation of European civility concepts. This process can be deeply analyzed in Mistral’s writings that shows us the continental transformation through the Mediterranean metaphor: between a Latin space and a space of miscegenation. In Mistral’s narratives, we can notice two kinds of analytical movements between North and South relations: when the writer talks about the European contrasts, she talks also about those of the American continent. In this context, the Old World, or its Southern part, shares its Historical experience with the New World to justify the positive perception of the New Latin American men: Multiethnic

Accessibility of Enzymatically Delignified Bambusa bambos for Efficient Hydrolysis at Minimum Cellulase Loading: An Optimization Study

In the present investigation, Bambusa bambos was used for optimization of enzymatic pretreatment and saccharification. Maximum enzymatic delignification achieved was 84%, after 8 h of incubation time. Highest reducing sugar yield from enzyme-pretreated Bambusa bambos was 818.01 mg/g dry substrate after 8 h of incubation time at a low cellulase loading (endoglucanase, β-glucosidase, exoglucanase, and xylanase were 1.63 IU/mL, 1.28 IU/mL, 0.08 IU/mL, and 47.93 IU/mL, respectively). Enzyme-treated substrate of Bambusa bambos was characterized by analytical techniques such as Fourier transformed infrared spectroscopy (FTIR), X-ray diffraction (XRD), and scanning electron microscopy (SEM). The FTIR spectrum showed that the absorption peaks of several functional groups were decreased after enzymatic pretreatment. XRD analysis indicated that cellulose crystallinity of enzyme-treated samples was increased due to the removal of amorphous lignin and hemicelluloses. SEM image showed that surface structure of Bambusa bambos was distorted after enzymatic pretreatment

Descending aortic calcification increases renal dysfunction and in-hospital mortality in cardiac surgery patients with intraaortic balloon pump counterpulsation placed perioperatively : a case control study

Introduction: Acute kidney injury (AKI) after cardiac surgery increases length of hospital stay and in-hospital mortality. A significant number of patients undergoing cardiac surgical procedures require perioperative intra-aortic balloon pump (IABP) support. Use of an IABP has been linked to an increased incidence of perioperative renal dysfunction and death. This might be due to dislodgement of atherosclerotic material in the descending thoracic aorta (DTA). Therefore, we retrospectively studied the correlation between DTA atheroma, AKI and in-hospital mortality. Methods: A total of 454 patients were retrospectively matched to one of four groups: -IABP/-DTA atheroma, +IABP/-DTA atheroma, -IABP/+DTA atheroma, +IABP/+DTA atheroma. Patients were then matched according to presence/absence of DTA atheroma, presence/absence of IABP, performed surgical procedure, age, gender and left ventricular ejection fraction (LVEF). DTA atheroma was assessed through standard transesophageal echocardiography (TEE) imaging studies of the descending thoracic aorta. Results: Basic patient characteristics, except for age and gender, did not differ between groups. Perioperative AKI in patients with -DTA atheroma/+IABP was 5.1% versus 1.7% in patients with -DTA atheroma/-IABP. In patients with +DTA atheroma/+IABP the incidence of AKI was 12.6% versus 5.1% in patients with +DTA atheroma/-IABP. In-hospital mortality in patients with +DTA atheroma/-IABP was 3.4% versus 8.4% with +DTA atheroma/+IABP. In patients with +DTA atheroma/+IABP in hospital mortality was 20.2% versus 6.4% with +DTA atheroma/-IABP. Multivariate logistic regression identified DTA atheroma > 1 mm (P = *0.002, odds ratio (OR) = 4.13, confidence interval (CI) = 1.66 to 10.30), as well as IABP support (P = *0.015, OR = 3.04, CI = 1.24 to 7.45) as independent predictors of perioperative AKI and increased in-hospital mortality. DTA atheroma in conjunction with IABP significantly increased the risk of developing acute kidney injury (P = 0.0016) and in-hospital mortality (P = 0.0001) when compared to control subjects without IABP and without DTA atheroma. Conclusions: Perioperative IABP and DTA atheroma are independent predictors of perioperative AKI and in-hospital mortality. Whether adding an IABP in patients with severe DTA calcification increases their risk of developing AKI and mortality postoperatively cannot be clearly answered in this study. Nevertheless, when IABP and DTA are combined, patients are more likely to develop AKI and to die postoperatively in comparison to patients without IABP and DTA atheroma

Measuring patient-perceived quality of care in US hospitals using Twitter

BACKGROUND: Patients routinely use Twitter to share feedback about their experience receiving healthcare. Identifying and analysing the content of posts sent to hospitals may provide a novel real-time measure of quality, supplementing traditional, survey-based approaches. OBJECTIVE: To assess the use of Twitter as a supplemental data stream for measuring patient-perceived quality of care in US hospitals and compare patient sentiments about hospitals with established quality measures. DESIGN: 404 065 tweets directed to 2349 US hospitals over a 1-year period were classified as having to do with patient experience using a machine learning approach. Sentiment was calculated for these tweets using natural language processing. 11 602 tweets were manually categorised into patient experience topics. Finally, hospitals with ≥50 patient experience tweets were surveyed to understand how they use Twitter to interact with patients. KEY RESULTS: Roughly half of the hospitals in the US have a presence on Twitter. Of the tweets directed toward these hospitals, 34 725 (9.4%) were related to patient experience and covered diverse topics. Analyses limited to hospitals with ≥50 patient experience tweets revealed that they were more active on Twitter, more likely to be below the national median of Medicare patients (p<0.001) and above the national median for nurse/patient ratio (p=0.006), and to be a non-profit hospital (p<0.001). After adjusting for hospital characteristics, we found that Twitter sentiment was not associated with Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) ratings (but having a Twitter account was), although there was a weak association with 30-day hospital readmission rates (p=0.003). CONCLUSIONS: Tweets describing patient experiences in hospitals cover a wide range of patient care aspects and can be identified using automated approaches. These tweets represent a potentially untapped indicator of quality and may be valuable to patients, researchers, policy makers and hospital administrators