AutoParallel: A Python module for automatic parallelization and distributed execution of affine loop nests

The last improvements in programming languages, programming models, and frameworks have focused on abstracting the users from many programming issues. Among others, recent programming frameworks include simpler syntax, automatic memory management and garbage collection, which simplifies code re-usage through library packages, and easily configurable tools for deployment. For instance, Python has risen to the top of the list of the programming languages due to the simplicity of its syntax, while still achieving a good performance even being an interpreted language. Moreover, the community has helped to develop a large number of libraries and modules, tuning them to obtain great performance. However, there is still room for improvement when preventing users from dealing directly with distributed and parallel computing issues. This paper proposes and evaluates AutoParallel, a Python module to automatically find an appropriate task-based parallelization of affine loop nests to execute them in parallel in a distributed computing infrastructure. This parallelization can also include the building of data blocks to increase task granularity in order to achieve a good execution performance. Moreover, AutoParallel is based on sequential programming and only contains a small annotation in the form of a Python decorator so that anyone with little programming skills can scale up an application to hundreds of cores.Comment: Accepted to the 8th Workshop on Python for High-Performance and Scientific Computing (PyHPC 2018

Coastal evolution in a Mediterranean microtidal zone: mid to late Holocene natural dynamics and human management of the Castelló lagoon, NE Spain

© 2016 Ejarque et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. We present a palaeoenvironmental study of the Castelló lagoon (NE Spain), an important archive for understanding long-term interactions between dynamic littoral ecosystems and human management. Combining geochemistry, mineralogy, ostracods, diatoms, pollen, non-pollen palynomorphs, charcoal and archaeo-historical datasets we reconstruct: 1) the transition of the lagoon from a marine to a marginal environment between ∼ 3150 cal BC to the 17 th century AD; 2) fluctuations in salinity; and 3) natural and anthropogenic forces contributing to these changes. From the Late Neolithic to the Medieval period the lagoon ecosystem was driven by changing marine influence and the land was mainly exploited for grazing, with little evidence for impact on the natural woodland. Land-use exploitation adapted to natural coastal dynamics, with maximum marine flooding hampering agropastoral activities between ∼ 1550 and ∼ 150 cal BC. In contrast, societies actively controlled the lagoon dynamics and become a major agent of landscape transformation after the Medieval period. The removal of littoral woodlands after the 8 th century was followed by the expansion of agrarian and industrial activities. Regional mining and smelting activities polluted the lagoon with heavy metals from the ∼ 11 th century onwards. The expansion of the milling industry and of agricultural lands led to the channelization of the river Muga into the lagoon after ∼ 1250 cal AD. This caused its transformation into a freshwater lake, increased nutrient load, and the infilling and drainage of a great part of the lagoon. By tracking the shift towards an anthropogenically-controlled system around∼ 750 yr ago, this study points out Mediterranean lagoons as ancient and heavily-modified systems, with anthropogenic impacts and controls covering multi-centennial and even millennial timescales. Finally, we contributed to the future construction of reliable seashell-based chronologies in NE Spain by calibrating the Banyuls-sur-Mer ▵ R offset with ceramic imports from the Emporiae archaeological site

Pitch production during the Roman period: an intensive mountain industry for a globalised economy?

The authors’ research project in the Pyrenees mountains has located and excavated Roman kilns for producing pitch from pine resin. Their investigations reveal a whole sustainable industry, integrated into the local environmental cycle, supplying pitch to the Roman network and charcoal as a spin-off to the local iron extractors. The paper makes a strong case for applying combined archaeological and palaeoenvironmental investigations in upland areas, showing mountain industries to have been not so much marginal and pastoral as key players in the economy of the Roman period and beyond it into the seventh century AD

Towards Automatic Application Migration to Clouds

Porting applications to Clouds is one of the key challenges in software industry. The available approaches to perform this task are basically either services derived from alliances of major software vendors and Cloud providers focusing on their own products, or small platform providers focusing on the most popular software stacks. For migrating other types of software, the options are limited to infrastructure-as-a-Service (IaaS) solutions which require a lot of programming effort for adapting the software to a Cloud provider’s API. Moreover, if it must be deployed in different providers, new integration procedures must be designed and implemented which could be a nightmare. This paper presents a solution for facilitating the migration of any application to the cloud, inferring the most suitable deployment model for the application and automatically deploying it in the available Cloud providers

A Programming Model for Hybrid Workflows: combining Task-based Workflows and Dataflows all-in-one

This paper tries to reduce the effort of learning, deploying, and integrating several frameworks for the development of e-Science applications that combine simulations with High-Performance Data Analytics (HPDA). We propose a way to extend task-based management systems to support continuous input and output data to enable the combination of task-based workflows and dataflows (Hybrid Workflows from now on) using a single programming model. Hence, developers can build complex Data Science workflows with different approaches depending on the requirements. To illustrate the capabilities of Hybrid Workflows, we have built a Distributed Stream Library and a fully functional prototype extending COMPSs, a mature, general-purpose, task-based, parallel programming model. The library can be easily integrated with existing task-based frameworks to provide support for dataflows. Also, it provides a homogeneous, generic, and simple representation of object and file streams in both Java and Python; enabling complex workflows to handle any data type without dealing directly with the streaming back-end.Comment: Accepted in Future Generation Computer Systems (FGCS). Licensed under CC-BY-NC-N

Synthesis of a Heterometallic [Zn2Ca] Pinwheel Array Stabilized by Amide-Amide Synthons

The rational design of heterometallic compounds bearing s-block metal ions have been a difficult task for chemists owing to their lack of preferential geometries. However, some strategies, such as the design of coordinating pockets with different sizes and/or donor atoms, have offered great results. In this work, this strategy has been tested using Ca(II) as an s-block metal ion and a compound previously obtained by our group with the formula [Zn(μ-ACA)(4-phpy)], which contains tetrahedral N,O- and octahedral O-coordinating pockets as a model structure. From this work, the corresponding heterometallic compound with the formula [ZnCa(μ-ACA)(4-phpy)]·EtOH (1) has been successfully synthesized, and fully characterized, and its crystal structure has been elucidated. Furthermore, we have compiled all the crystal structures containing [ZnM] pinwheel secondary building units (SBUs), where M stands for an s-block metal ion, and the observed tendencies, as well as the promising applications as template SBUs for the preparation of 1D-3D coordination polymers, have been discussed. Finally, solid-state UV-Vis and photoluminescence have been recorded and compared with the homometallic [Zn(μ-ACA)(4-phpy)] compound

Service Orchestration on a Heterogeneous Cloud Federation

During the last years, the cloud computing technology has emerged as a new way to obtain computing resources on demand in a very dynamic fashion and only paying for what you consume. Nowadays, there are several hosting providers which follow this approach, offering resources with different capabilities, prices and SLAs. Therefore, depending on the users' preferences and the application requirements, a resource provider can fit better with them than another one. In this paper, we present an architecture for federating clouds, aggregating resources from different providers, deciding which resources and providers are the best for the users' interests, and coordinating the application deployment in the selected resources giving to the user the impression that a single cloud is used

Contrast-enhanced mammography-guided biopsy: technical feasibility and first outcomes

Objectives To evaluate the feasibility of contrast-enhanced mammography (CEM)-guided biopsy at Hospital del Mar, a Spanish university hospital. Methods We retrospectively reviewed all consecutive women with a suspicious enhancing finding eligible for CEM-guided biopsy, who were prospectively enrolled in a pre-marketing clinical validation and feasibility study (October 2019 to September 2021). CEM-guided biopsy is a stereotactic-based procedure that, by using intravenous iodinated contrast media administration and dual-energy acquisition, provides localisation of enhancing lesions. All the biopsies were performed using a vacuum-assisted device. We collected procedural characteristics (patient position and type of approach), and histopathological results. Feasibility endpoints included success (visualisation of the enhancing lesion, post-procedural biopsy changes and clip placement), procedural time, number of scout acquisitions and complications. Results A total of 66 suspicious enhancing lesions (18.0% foci, 44.0% mass, 38.0% non-mass enhancement; median size 8.5 mm) in 64 patients (median age 59 years, mostly minimal [48.4%] or mild [32.8%] background parenchymal enhancement) were referred for CEM-guided biopsy in the study period. The success rate was 63/66 (95.4%). Amongst successful procedures, patients were most frequently seated (52/63, 82.5%) and the preferred approach was horizontal (48/63, 76.2%). Median total time per procedure was 15 min. Median number of acquisitions needed before targeting was 2 (range 1-4). Complications consisted of hematoma (17/63, 27%) and vasovagal reaction (2/63, 3.2%). At histology, the malignancy rate was 25/63 (39.7%). Conclusion In this first patient series, CEM-guided breast biopsy was feasible, with success and complication rates similar to those previously reported for magnetic resonance guidance

Semantic resource allocation with historical data based predictions

One of the most important issues for Service Providers in Cloud Computing is delivering a good quality of service. This is achieved by means of the adaptation to a changing environment where different failures can occur during the execution of different services and tasks. Some of these failures can be predicted taking into account the information obtained from previous executions. The results of these predictions will help the schedulers to improve the allocation of resources to the different tasks. In this paper, we present a framework which uses semantically enhanced historical data for predicting the behavior of tasks and resources in the system, and allocating the resources according to these predictions

Enabling System Wide Shared Memory for Performance Improvement in PyCOMPSs Applications

Python has been gaining some traction for years in the world of scientific applications. However, the high-level abstraction it provides may not allow the developer to use the machines to their peak performance. To address this, multiple strategies, sometimes complementary, have been developed to enrich the software ecosystem either by relying on additional libraries dedicated to efficient computation (e.g., NumPy) or by providing a framework to better use HPC scale infrastructures (e.g., PyCOMPSs).In this paper, we present a Python extension based on SharedArray that enables the support of system-provided shared memory and its integration into the PyCOMPSs programming model as an example of integration to a complex Python environment. We also evaluate the impact such a tool may have on performance in two types of distributed execution-flows, one for linear algebra with a blocked matrix multiplication application and the other in the context of data-clustering with a k-means application. We show that with very little modification of the original decorator (3 lines of code to be modified) of the task-based application the gain in performance can rise above 40% for tasks relying heavily on data reuse on a distributed environment, especially when loading the data is prominent in the execution time.This work was partly funded by the EXPERTISE project (http://www.msca-expertise.eu/), which has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 721865. BSC authors have also been supported by the Spanish Government through contracts SEV2015-0493 and TIN2015-65316-P, and by Generalitat de Catalunya through contract 2014-SGR-1051.Peer ReviewedPostprint (author's final draft