From Cluster to Grid: A Case Study in Scaling-Up a Molecular Electronics Simulation Code

This paper describes an ongoing project whose goal is to significantly improve the performance and applicability of a molecular electronics simulation code. The specific goals are to (1) increase computational performance on the simulation problems currently being solved by our physics collaborators; (2) allow much larger problems to be solved in reasonable time; and (3) expand the set of resources available to the code, from a single homogeneous cluster to a campus-wide computational grid, while maintaining acceptable performance across this larger set of resources. We describe the sequential performance of the code, the performance of two parallel versions, and the benefits of problem-specific load balancing strategies. The grid context motivates the need for runtime algorithm selection; we present a component-based software framework that makes this possible

Automated component creation for legacy C++ and fortran codes.

A significant amount of work has been spent creating component models and programming environments, but little support exists for automation in the process of creating components from existing codes. To entice users to adopt the component-based paradigm over traditional programming models, integration of legacy codes must be as simple and fast as possible, We present a system for automating the IDL generation stage of component development based on source code analysis of legacy C, C-t-4 and Fortran codes using the Program Database Toolkit. Together with IDL compilation tools such as Babel, we provide an alternative to hand-written IDL code for legacy applications and libraries. In addition to generating IDL, we propose an XML-based method for specifying meta-data related to type mapping and wrapper generation that can be shared between our tools and IDL compilers. The component model of choice for this work is the Common Component Architecture (CCA) using the Scientific Interface Definition Language (SIDL), though the concepts presented can be applied to other models

Desarrollo de un interfaz gráfica remota para un caracterizador de semiconductores y un analizador de redes

This thesis projects presents the development of an application for the control and automatization of a Keithley K4200 Semiconductor Characterization System and and Agilent E5071c Vector Network Analyzer. The application was developed in four different parts. Everything was implemented using the Python programming language The first part is a library that implements general transmission of commands to the instruments, as well as recovery of the data produced by a measurements. Everything is done over TCP/IP. The second part involves an object oriented library used to validate and generate commands to make measurements with the Keithley K4200-SCS. This library exposes the basic functionality of the device. The third part involves an object oriented library used to validate and generate commands to make measurements with the Agilent E5071c. The fourth part is a graphical user interface that uses the previously developed libraries to present a unified interface for the Keithely K4200-SCS and the Agilent E5071c. The library used for this development was PyQt.Este proyecto de tesis presenta el desarrollo de una aplicación para el control y automatización de un Sistema Caracterizador de Semiconductores Keithley K4200-SCS y de un Analizador de Redes Agilent E5071c. El software se desarrolló en cuatros partes diferentes. Todas implementadas utilizando el lenguaje de programación Python. La primera parte es una librería de comunicación general que se encarga de enviar comandos a los instrumentos y de recuperar los datos generados por una medición. Todo a través del protocolo TCP/IP. La segunda parte es una librería orientada a objetos que valida y genera comandos para realizar mediciones en el Keithley K4200-SCS. Esta librería expone la funcionalidad básica del caracterizador de semiconductores. La tercera parte es una librería orientada a objetos que valida y genera comandos para el Agilent E5071c. La cuarta parte consta de una interfaz gráfica que hace uso de estas librerías para presentar una interfaz unificada para el Agilent E5071c y el Keithley K4200-SCS. La librería utilizada para la interfaz gráfica fue PyQt

Überblick zur Softwareentwicklung in Wissenschaftlichen Anwendungen

Viele wissenschaftliche Disziplinen müssen heute immer komplexer werdende numerische Probleme lösen. Die Komplexität der benutzten wissenschaftlichen Software steigt dabei kontinuierlich an. Diese Komplexitätssteigerung wird durch eine ganze Reihe sich ändernder Anforderungen verursacht: Die Betrachtung gekoppelter Phänomene gewinnt Aufmerksamkeit und gleichzeitig müssen neue Technologien wie das Grid-Computing oder neue Multiprozessorarchitekturen genutzt werden, um weiterhin in angemessener Zeit zu Berechnungsergebnissen zu kommen. Diese Fülle an neuen Anforderungen kann nicht mehr von kleinen spezialisierten Wissenschaftlergruppen in Isolation bewältigt werden. Die Entwicklung wissenschaftlicher Software muss vielmehr in interdisziplinären Gruppen geschehen, was neue Herausforderungen in der Softwareentwicklung induziert. Ein Paradigmenwechsel zu einer stärkeren Separation von Verantwortlichkeiten innerhalb interdisziplinärer Entwicklergruppen ist bis jetzt in vielen Fällen nur in Ansätzen erkennbar. Die Kopplung partitioniert durchgeführter Simulationen physikalischer Phänomene ist ein wichtiges Beispiel für softwaretechnisch herausfordernde Aufgaben im Gebiet des wissenschaftlichen Rechnens. In diesem Kontext modellieren verschiedene Simulationsprogramme unterschiedliche Teile eines komplexeren gekoppelten Systems. Die vorliegende Arbeit gibt einen Überblick über Paradigmen, die darauf abzielen Softwareentwicklung für Berechnungsprogramme verlässlicher und weniger abhängig voneinander zu machen. Ein spezielles Augenmerk liegt auf der Entwicklung gekoppelter Simulationen.Fields of modern science and engineering are in need of solving more and more complex numerical problems. The complexity of scientific software thereby rises continuously. This growth is caused by a number of changing requirements. Coupled phenomena gain importance and new technologies like the computational Grid, graphical and heterogeneous multi-core processors have to be used to achieve high-performance. The amount of additional complexity can not be handled by small groups of specialised scientists. The interdiciplinary nature of scientific software thereby presents new challanges for software engineering. A paradigm shift towards a stronger separation of concerns becomes necessary in the development of future scientific software. The coupling of independently simulated physical phenomena is an important example for a software-engineering concern in the domain of computational science. In this context, different simulation-programs model only a part of a more complex coupled system. The present work gives overview on paradigms which aim at making software-development in computational sciences more reliable and less interdependent. A special focus is put on the development of coupled simulations

Contribution à la conception à base de composants logiciels d'applications scientifiques parallèles

La conception d'applications scientifiques à base de couplage de code est une tâche complexe car elle demande de concilier une facilité de programmation et une obtention de haute performance. En outre, les ressources matérielles (supercalculateurs, grappes de calcul, grilles) permettant leur exécution forment un ensemble hétérogène en constante évolution. Les modèles à base de composants logiciels forment une piste prometteuse pour gérer ces deux sources de complexité car ils permettent d exprimer les interactions entre les différents constituants d une application tout en offrant des possibilités d'abstraction des ressources. Néanmoins, les modèles existants ne permettent pas d'exprimer de manière satisfaisante les applications constituées de motifs répliqués dynamiques et hiérarchiques. Ainsi, cette thèse vise à améliorer l'existant et en particulier la plate-forme générique de simulation numérique SALOME pour une classe d'applications très répandue : les applications à base de décomposition de domaine et la variante utilisant le raffinement de maillage adaptatif. Tout d'abord, nous avons proposé d étendre le modèle de composition spatial et temporel de SALOME en ajoutant la possibilité de définir dynamiquement la cardinalité des composants. Cela demande en particulier de gérer les communications de groupes ainsi induites. La proposition a été implémentée dans SALOME et validée via une application de décomposition de domaine à base de couplage de plusieurs instances de Code_Aster. Ensuite, nous avons étudié la pertinence d'utiliser un modèle de composant supportant des connecteurs natifs (MPI, mémoire partagée, appel de méthode) pour permettre une composition plus fine des interactions entre composants. Les résultats d'expériences montrent que des performances équivalentes aux versions natives sont obtenues tout en permettant de manipuler facilement l'architecture de l'application. Enfin, nous avons étudié les extensions nécessaires aux modèles à composants (abstraction,hiérarchie, dynamicité) pour la conception d applications de raffinement de maillage adaptatif. Les modèles de composants spatio-temporels les plus avancés permettent ainsi d'exprimer ce type d'application mais les performances sont limitées par leur mise en œuvre centralisée ainsi que par le manque de moyens efficaces pour modifier à la volée des assemblages de composants.Designing scientific applications based on code coupling is a complex task. It requires both an easy programming process and high-performance. In addition, execution resources (supercomputers, computer clusters, grids) are heterogeneous and constantly evolving. Software components models offer a promising perspective to manage this double complexity because they can express interactions between the different parts of an application while providing abstraction of resources. However, existing models cannot accurately express the applications made of dynamic and hierarchical patterns. The aim of this thesis is to improve the existing models, and in particular the generic platform for numerical simulation SALOME, for a class of widespread applications : applications based on domain decomposition, and its dynamic variant using adaptive mesh refinement. Firstly, we proposed to extend the spatial and temporal composition model provided by SALOME, by adding the ability to dynamically set component cardinality. This requires in particular to manage group communications induced. The proposal has been implemented into SALOME and validated via a domain decomposition application based on coupling several instances of Code_Aster.Then, we have studied the relevance of using a component model supporting native connectors (MPI, shared memory, method invocation), in order to allow finer composition interactions between components.The experiment results show that performances obtained are equivalent to those of the native versions, while allowing to easily manipulate the application architecture. Finally, we studied the necessary component models extensions (abstraction, hierarchy, dynamicity) for designing adaptative mesh refinement applications. The most advanced spatio-temporal component models can express this type of application but performances are limited by their centralized implementation and by the lack of efficient ways of modifying component assembling at execution time.LYON-ENS Sciences (693872304) / SudocSudocFranceF

Foundations for Automatic, Adaptable Compilation

Computational science demands extreme performance because the running time of an application often determines the size of the experiment that a scientist can reasonably compute. Unfortunately, traditional compiler technology is ill-equipped to harness the full potential of today's computing platforms, forcing scientists to spend time manually tuning their application's performance. Although improving compiler technology should alleviate this problem, two challenges obstruct this goal: hardware platforms are rapidly changing and application software is difficult to statically model and predict. To address these problems, this thesis presents two techniques that aim to improve a compiler's adaptability: automatic resource characterization and selective, dynamic optimization. Resource characterization empirically measures a system's performance-critical characteristics, which can be provided to a parameterized compiler that specializes programs accordingly. Measuring these characteristics is important, because a system's physical characteristics do not always match its observed characteristics. Consequently, resource characterization provides an empirical performance model of a system's actual behavior, which is better suited for guiding compiler optimizations than a purely theoretical model. This thesis presents techniques for determining a system's data cache and TLB capacity, line size, and associativity, as well as instruction-cache capacity. Even with a perfect architectural-model, compilers will still often generate suboptimal code because of the difficulty in statically analyzing and predicting a program's behavior. This thesis presents two techniques that enable selective, dynamic-optimization for cases in which static compilation fails to deliver adequate performance. First, intermediate-representation (IR) annotation generates a fully-optimized native binary tagged with a higher-level compiler representation of itself. The native binary benefits from static optimization and code generation, but the IR annotation allows targeted and aggressive dynamic-optimization. Second, adaptive code-selection allows a program to empirically tune its performance throughout execution by automatically identifying and favoring the best performing variant of a routine. This technique can be used for dynamically choosing between different static-compilation strategies; or, it can be used with IR annotation for performing dynamic, feedback-directed optimization