Secure, performance-oriented data management for nanoCMOS electronics

The EPSRC pilot project Meeting the Design Challenges of nanoCMOS Electronics (nanoCMOS) is focused upon delivering a production level e-Infrastructure to meet the challenges facing the semiconductor industry in dealing with the next generation of ‘atomic-scale’ transistor devices. This scale means that previous assumptions on the uniformity of transistor devices in electronics circuit and systems design are no longer valid, and the industry as a whole must deal with variability throughout the design process. Infrastructures to tackle this problem must provide seamless access to very large HPC resources for computationally expensive simulation of statistic ensembles of microscopically varying physical devices, and manage the many hundreds of thousands of files and meta-data associated with these simulations. A key challenge in undertaking this is in protecting the intellectual property associated with the data, simulations and design process as a whole. In this paper we present the nanoCMOS infrastructure and outline an evaluation undertaken on the Storage Resource Broker (SRB) and the Andrew File System (AFS) considering in particular the extent that they meet the performance and security requirements of the nanoCMOS domain. We also describe how metadata management is supported and linked to simulations and results in a scalable and secure manner

Optimization strategies for large-scale distributed computing and data management in the presence of security and other requirements

The growth of e-science and grid computing has presented new opportunities to researchers working on distributed, collaborative projects, or for whom access to large computational and data-storage resources is essential. There is, however, much work to be done to improve the experience for these users, in particular by mitigating the effects that failing to consider fully the requirements of their processing tasks (or jobs) can have on the likelihood of success. Any improvement in this regard would be significant, not only because of the beneficial effect it would have on useful resource utilization, but also due to the improved perception of grid computing it would engender among its users. Allocating jobs to resources is never easy, and this is particularly true when the resources are of heterogeneous construction, geographically distant, and owned and operated by a variety of institutions. Attempts to allocate jobs intelligently must consider not only the strict hardware requirements of the job and whether these match a particular candidate resource, but must also consider requirements related to the security infrastructures in place (including, for example, the rights of the user who wishes to run the job, the licences associated with the application to be run and the data it will use and produce), how the job might affect other users of the resource, and indeed how the behaviour of other users of the resource might affect the job. This thesis examines the data, security and legal requirements of typical collaborative research projects, and discusses how these requirements suggest particular constraints that will govern any jobs which need to be executed. It reviews some of the technologies which are common in this field, and describes current state-of-the-art technology in this area. This thesis then presents a framework for describing requirements, along with an algorithmic method for allocating jobs to resources. Case studies and an analysis of performance are presented using these algorithms, which show how they build upon, and improve, the state-of-the-art in this domain

Estudo de casos com aplicações científicas de alto desempenho em agregados de computadores multi-core

Dissertação (mestrado) - Universidade Federal de Santa Catarina, Centro Tecnológico. Programa de Pós-graduação em Ciência da ComputaçãoEste trabalho de disserta»c~ao concentra seu esfor»co no estudo emp¶³rico de tr^es casos com ambientes de cluster distintos e homog^eneos, sendo que dois deles s~ao ambientes operacionais de empresas. Tais agregados s~ao compostos por computadores multiprocessados com processadores mono-core e multi-core (dual-core e quad-core), interconectados por redes Gigabit Ethernet, e outro ambiente interconectado por uma rede Myrinet. O primeiro estudo de caso foi realizado em um ambiente experimental como um exerc¶³cio emp¶³rico sobre a rela»c~ao entre tecnologias de rede de interconex~ao e carac- ter¶³sticas inerentes a aplica»c~oes paralelas. Com isso, pretendeu-se entrar em contato com a realidade da computa»c~ao paralela por meio de arquiteturas paralelas distribu¶³das como os agregados de computadores. Al¶em disso, pode-se conhecer e analisar as diferen»cas em desempenho oferecido por sistemas de comunica»c~ao distintos, como a tecnologia de rede Myrinet face µa tecnologia Ethernet, diante de aplica»c~oes de granularidades distintas, bem como compreender as m¶etricas comumente adotadas em avalia»c~oes de desempenho. Dentre as contribui»c~oes do trabalho de pesquisa e experimenta»c~ao desenvolvido est¶a a redu»c~ao do tempo de execu»c~ao de aplica»c~oes cient¶³¯cas grand challenge, como a modelagem num¶erica de previs~ao meteorol¶ogica. Sendo assim, busca-se como resultado a otimiza»c~ao de desempenho do ambiente de cluster em compara»c~ao µa sua condi»c~ao anterior, sem nenhuma especializa»c~ao µa aplica»c~ao em foco. Nesse sentido, dois estudos de casos foram realizados em agregados de computadores pertencentes a organiza»c~oes, em uma aproxima»c~ao com a realidade da computa»c~ao de alto desempenho em ambientes de produ»c~ao. Com a realiza»c~ao deste estudo emp¶³rico como um todo, pode-se contrastar na pr¶atica os pontos estudados durante a revis~ao bibliogr¶a¯ca. Foi poss¶³vel compreender melhor as vantagens e desvantangens envolvidas nesses ambientes enquanto sistemas paralelos distribu¶³dos, com o foco voltado µa modelagem de sistemas de alto desempenho em ambientes de produ»c~ao. Durante o processo de otimiza»c~ao do desempenho, entrou-se em contato com os mecanismos de intera»c~ao entre processos e os modelos de programa»c~ao paralela envolvidos ao mesmo tempo em que investigou-se o impacto da tend^encia atual no que diz respeito a processadores multi-core, bem como os fatores redutores do desempenho (que resultam em overhead). En¯m, o conhecimento adquirido com os estudos de casos possibilita uma melhor compreens~ao do processo e dos fatores envolvidos na implementa»c~ao de ambientes de cluster adequados a cada aplica»c~ao paralela com demanda por alto desempenho, a ¯m de aproveitar melhor os recursos agregados. Al¶em disso, a import^ancia deste trabalho transcende µa ci^encia da computa»c~ao como disciplina acad^emica, pois a empresa parceira ganha em capacidade e qualidade na previs~ao meteorol¶ogica do tempo, seja para prevenir o impacto de desastres naturais ou para auxiliar na produ»c~ao agr¶³cola, e tamb¶em em potencial de pesquisa no ^ambito daquela ¶area de atua»c~ao. This dissertation concentrates its e®ort on the empirical study of three cases with distinct and homogeneous cluster con¯gurations, two of them operational environments at organizations. Theses clusters are equipped with multiprocessor computers, including multiple single-core and multi-core processors (dual-core and quad-core), interconnected by Gigabit Ethernet networks, and one environment interconnected with a Myrinet device. The ¯rst case study was performed on an experimental environment as an empiri-cal exercise about the relationship between interconnect technologies and characteristics inherent to parallel applications, in order to get in touch with the reality of parallel computing through parallel distributed architectures such as a cluster. Furthermore, we could acknowledge and analyze the di®erences in performance o®ered by di®erent communication systems, opposing Myrinet and Ethernet networking technologies before applications of di®erent granularity, as well as understand common metrics adopted for performance assessments. One of the contributions of this empirical and research work is to reduce the wall clock (or elapsed) time of grand challenge scienti¯c applications, such as numerical weather prediction models. Therefore, it should result in a better performance of the cluster environment compared to its previous condition, with no adaptation for the running application. Based on that, two case studies were conducted on operational clusters belonging to organizations in order to interact with the reality of high performance computing in production environments. Performing this empirical study as a whole confronts the knowledge obtained throughout the literature review putting them into practice. Moreover, we could accomplish a better understanding of the trade-o®s involved in cluster environments as distributed parallel systems for production environments from the point of view of an architectural designer. During this optimization process, we could understand the mechanisms for pro-cesses interaction and parallel programming models as well as the factors for overhead growth and performance reduction. Finally, the knowledge acquired with these case studies allow us to better comprehend the process and the factors involved in the implementation and adaptation of cluster en vironments to a speci¯c high performance application, in order to better employ the aggregated computing resources. Furthermore, the importance of this work transcends computer sciences as an academic subject, because the partner organization gains capacity and quality for predicting weather conditions, either to prevent us from the impact of natural disasters or to enhance agricultural production, as well as gains in research potential within that speci¯c area

3rd EGEE User Forum

We have organized this book in a sequence of chapters, each chapter associated with an application or technical theme introduced by an overview of the contents, and a summary of the main conclusions coming from the Forum for the chapter topic. The first chapter gathers all the plenary session keynote addresses, and following this there is a sequence of chapters covering the application flavoured sessions. These are followed by chapters with the flavour of Computer Science and Grid Technology. The final chapter covers the important number of practical demonstrations and posters exhibited at the Forum. Much of the work presented has a direct link to specific areas of Science, and so we have created a Science Index, presented below. In addition, at the end of this book, we provide a complete list of the institutes and countries involved in the User Forum