Spark versus Flink: Understanding Performance in Big Data Analytics Frameworks

International audienceBig Data analytics has recently gained increasing popularity as a tool to process large amounts of data on-demand. Spark and Flink are two Apache-hosted data analytics frameworks that facilitate the development of multi-step data pipelines using directly acyclic graph patterns. Making the most out of these frameworks is challenging because efficient executions strongly rely on complex parameter configurations and on an in-depth understanding of the underlying architectural choices. Although extensive research has been devoted to improving and evaluating the performance of such analytics frameworks, most of them benchmark the platforms against Hadoop, as a baseline, a rather unfair comparison considering the fundamentally different design principles. This paper aims to bring some justice in this respect, by directly evaluating the performance of Spark and Flink. Our goal is to identify and explain the impact of the different architectural choices and the parameter configurations on the perceived end-to-end performance. To this end, we develop a methodology for correlating the parameter settings and the operators execution plan with the resource usage. We use this methodology to dissect the performance of Spark and Flink with several representative batch and iterative workloads on up to 100 nodes. Our key finding is that there none of the two framework outperforms the other for all data types, sizes and job patterns. This paper performs a fine characterization of the cases when each framework is superior, and we highlight how this performance correlates to operators, to resource usage and to the specifics of the internal framework design

Parallel and Distributed Data Management. Introduction

The manipulation and handling of an ever increasing volume of data by current data-intensive applications require novel techniques for e?cient data management. Despite recent advances in every aspect of data management (storage, access, querying, analysis, mining), future applications are expected to scale to even higher degrees, not only in terms of volumes of data handled but also in terms of users and resources, often making use of multiple, pre-existing autonomous, distributed or heterogeneous resources

Pinturas rupestres en el Barranc de Carbonera (Beniatjar, Valencia). Nuevas lecturas de un yacimiento excepcional

La revisión de las imágenes pintadas en los dos abrigos del Barranc de Carbonera, en Beniatjar (Valencia), y el descubrimiento de nuevos e importantes conjuntos con arte rupestre en la Vall d’Albaida, permite replantear el origen y difusión del Arte Esquemático en el Arco mediterráneo de la Península Ibérica y su relación con el denominado Arte Esquemático Antiguo en las tierras valencianas

Tris[N-(2-furoyl)-N,N′-diphenyl­thio­ureato-κ2 O,S]cobalt(III)

In the title compound, [Co(C18H13N2O2S)3], the CoIII atom is coordinated by the S and O atoms of three N-furoyl-N′,N′-diphenyl­thio­urea ligands in a slightly distorted octa­hedral geometry. The three O atoms are arranged fac, as are the three S atoms

cis-Bis[N-(2-furoyl)-N′,N′-diphenyl­thio­ureato-κ2 O,S]nickel(II)

In the title compound, [Ni(C18H13N2O2S)2], the NiII atom is coordinated by the S and O atoms of two N-furoyl-N′,N′-diphenyl­thio­ureate ligands in a slightly distorted square-planar coordination geometry. The two O and two S atoms are cis to each other

Asociación obesidad e hiperplasia nodular focal (HNF) telangiectásica. Reevaluación de 24 casos.

La hiperplasia nodular focal no es una verdadera neo plasia. Es una respuesta regenerativa de los hepatocitos a una anomalía vascular. Se reevaluaron 24 casos diagnosticados y confirmados en el estudio anatomopatológico como hiperplasia nodular focal. Tres de los 24 casos fueron reclasificados como adenomas inflamatorios telangiectásico, vinculado con antecedentes de síndromes metabolicos y con imnumorreactividad frente a la Amiloide A. La presencia de ectasia vascular, dilatación sinusoidal, áreas de peliosis con signos inflamatorios focales o difusos asociados a inmunorreactividad frente al Amiloide A son signos histológicos que nos indican la presencia de adenomas hepatocelulares inflamatorios telangiectásicos antiguamente clasificados como hiperplasia nodular focal atípica. El antecedentes de obesidad u hígado graso, unido al incremento en la reactividad frente al Amiloide A caracteriza a los adenomas inflamatorios telan giec - tásicos

High On/Off ratio memristive switching of manganite/cuprate bilayer by interfacial magnetoelectricity

Memristive switching serves as the basis for a new generation of electronic devices. Memristors are two-terminal devices in which the current is turned on and off by redistributing point defects, e.g., vacancies, which is difficult to control. Memristors based on alternative mechanisms have been explored, but achieving both the high On/Off ratio and the low switching energy desirable for use in electronics remains a challenge. Here we report memristive switching in a La_(0.7)Ca_(0.3)MnO_(3)/PrBa_(2)Cu_(3)O_(7) bilayer with an On/Off ratio greater than 103 and demonstrate that the phenomenon originates from a new type of interfacial magnetoelectricity. Using results from firstprinciples calculations, we show that an external electric-field induces subtle displacements of the interfacial Mn ions, which switches on/off an interfacial magnetic “dead” layer, resulting in memristive behavior for spin-polarized electron transport across the bilayer. The interfacial nature of the switching entails low energy cost about of a tenth of atto Joule for write/erase a “bit”. Our results indicate new opportunities for manganite/cuprate systems and other transition-metal-oxide junctions in memristive applications

A Semantic-Based Approach to Attain Reproducibility of Computational Environments in Scientific Workflows: A Case Study

Abstract. Reproducible research in scientific workflows is often addressed by tracking the provenance of the produced results. While this approach allows inspecting intermediate and final results, improves understanding, and permits replaying a workflow execution, it does not ensure that the computational environment is available for subsequent executions to reproduce the experiment. In this work, we propose describing the resources involved in the execution of an experiment using a set of semantic vocabularies, so as to conserve the computational environment. We define a process for documenting the workflow application, management system, and their dependencies based on 4 domain ontologies. We then conduct an experimental evaluation using a real workflow application on an academic and a public Cloud platform. Results show that our approach can reproduce an equivalent execution environment of a predefined virtual machine image on both computing platforms

A Semantic-Based Approach to Attain Reproducibility of Computational Environments in Scientific Workflows: A Case Study

Abstract. Reproducible research in scientific workflows is often addressed by tracking the provenance of the produced results. While this approach allows inspecting intermediate and final results, improves understanding, and permits replaying a workflow execution, it does not ensure that the computational environment is available for subsequent executions to reproduce the experiment. In this work, we propose describing the resources involved in the execution of an experiment using a set of semantic vocabularies, so as to conserve the computational environment. We define a process for documenting the workflow application, management system, and their dependencies based on 4 domain ontologies. We then conduct an experimental evaluation using a real workflow application on an academic and a public Cloud platform. Results show that our approach can reproduce an equivalent execution environment of a predefined virtual machine image on both computing platforms

Análisis epidemiológico de los fallecimientos por suicidio, Uruguay año 2000

Con el objetivo de conocer las principales características epidemiológicas de la mortalidad por suicidio en el Uruguay, se realizó un estudio descriptivo. Uruguay se ubicó en el primer tercio de los países con las tasas de suicidio más elevadas del mundo, en el tercer lugar de América y en el primero de Sudamérica. Presentó para el año 2000 una tasa del 18.76 por cien mil, ocurriendo un promedio de 1.7 suicidios por día, representando la novena causa de muerte. Las tasas más elevadas se presentaron en el sexo masculino; entre divorciados/as y viudos/as y el método más usado fue el disparo por arma de fuego. Para el Departamento de Montevideo, el promedio de AVPP por suicidio fue de 28 y 29 años para el sexo femenino y masculino respectivamente, se destaca las mayores tasas en el nivel de instrucción terciaria, en desocupados y jubilados; y en niveles socioeconómicos medios o bajos. El suicidio es considerado un problema de Salud Pública