Partitioning SKA Dataflows for Optimal Graph Execution

Optimizing data-intensive workflow execution is essential to many modern scientific projects such as the Square Kilometre Array (SKA), which will be the largest radio telescope in the world, collecting terabytes of data per second for the next few decades. At the core of the SKA Science Data Processor is the graph execution engine, scheduling tens of thousands of algorithmic components to ingest and transform millions of parallel data chunks in order to solve a series of large-scale inverse problems within the power budget. To tackle this challenge, we have developed the Data Activated Liu Graph Engine (DALiuGE) to manage data processing pipelines for several SKA pathfinder projects. In this paper, we discuss the DALiuGE graph scheduling sub-system. By extending previous studies on graph scheduling and partitioning, we lay the foundation on which we can develop polynomial time optimization methods that minimize both workflow execution time and resource footprint while satisfying resource constraints imposed by individual algorithms. We show preliminary results obtained from three radio astronomy data pipelines.Comment: Accepted in HPDC ScienceCloud 2018 Worksho

Lower bounds for dilation, wirelength, and edge congestion of embedding graphs into hypercubes

Interconnection networks provide an effective mechanism for exchanging data between processors in a parallel computing system. One of the most efficient interconnection networks is the hypercube due to its structural regularity, potential for parallel computation of various algorithms, and the high degree of fault tolerance. Thus it becomes the first choice of topological structure of parallel processing and computing systems. In this paper, lower bounds for the dilation, wirelength, and edge congestion of an embedding of a graph into a hypercube are proved. Two of these bounds are expressed in terms of the bisection width. Applying these results, the dilation and wirelength of embedding of certain complete multipartite graphs, folded hypercubes, wheels, and specific Cartesian products are computed

Mapping en arquitecturas heterogéneas

En esta Tesis Doctoral se definieron dos modelos basados en grafos para la representación de aplicaciones paralelas (TTIGHA, MPAHA) las cuales pueden ser ejecutadas en arquitecturas homogéneas o heterogéneas. A su vez, en esta Tesis también se implementaro dos algoritmos de mapping (MATEHA, MATEHAIB) basados en el primer modelo y otro algoritmo de mapping (AMTHA) basado en el segundo modelo.Eje: Concurso de tesisRed de Universidades con Carreras en Informática (RedUNCI

Mapping en arquitecturas heterogéneas

En esta Tesis Doctoral se definieron dos modelos basados en grafos para la representación de aplicaciones paralelas (TTIGHA, MPAHA) las cuales pueden ser ejecutadas en arquitecturas homogéneas o heterogéneas. A su vez, en esta Tesis también se implementaro dos algoritmos de mapping (MATEHA, MATEHAIB) basados en el primer modelo y otro algoritmo de mapping (AMTHA) basado en el segundo modelo.Eje: Concurso de tesisRed de Universidades con Carreras en Informática (RedUNCI

Software radios: unifying the reconfiguration process over heterogeneous platforms

Future radio transceivers supporting the software radio concept will provide increased features for radio access networks. However, the reconfiguration of radio equipment requires the existence of architecture, a common framework, which allows the flexible management of software running on radio processors. Such a framework must take into account the heterogeneity of hardware devices and platforms for radio applications. Since the flexibility has a cost in terms of added overhead, a conceptually simple but efficient structure that allows powerful mechanisms to develop and deploy software radio applications is required. This paper describes our approach, the reasons that motivated it, and some implementation issues. The proposed framework is essentially based on four items: an abstraction layer which hides any platform-dependent issue, a simple time-driven software structure, a delimited interface format for software blocks which does not actually constrain communication, and a global time-reference mechanism to guarantee real-time behavior.Peer Reviewe

Parallelization of Finite Element Analysis Codes Using Heterogeneous Distributed Computing

Performance gains in computer design are quickly consumed as users seek to analyze larger problems to a higher degree of accuracy. Innovative computational methods, such as parallel and distributed computing, seek to multiply the power of existing hardware technology to satisfy the computational demands of large applications. In the early stages of this project, experiments were performed using two large, coarse-grained applications, CSTEM and METCAN. These applications were parallelized on an Intel iPSC/860 hypercube. It was found that the overall speedup was very low, due to large, inherently sequential code segments present in the applications. The overall execution time T(sub par), of the application is dependent on these sequential segments. If these segments make up a significant fraction of the overall code, the application will have a poor speedup measure