Scientific Workflow Applications on Amazon EC2

The proliferation of commercial cloud computing providers has generated significant interest in the scientific computing community. Much recent research has attempted to determine the benefits and drawbacks of cloud computing for scientific applications. Although clouds have many attractive features, such as virtualization, on-demand provisioning, and "pay as you go" usage-based pricing, it is not clear whether they are able to deliver the performance required for scientific applications at a reasonable price. In this paper we examine the performance and cost of clouds from the perspective of scientific workflow applications. We use three characteristic workflows to compare the performance of a commercial cloud with that of a typical HPC system, and we analyze the various costs associated with running those workflows in the cloud. We find that the performance of clouds is not unreasonable given the hardware resources provided, and that performance comparable to HPC systems can be achieved given similar resources. We also find that the cost of running workflows on a commercial cloud can be reduced by storing data in the cloud rather than transferring it from outside

Infrastructure Specifications

This document presents the computing infrastructure deployed in the context of StratusLab project, details the configuration of the computing resources used and the commitments made from the project partners to contribute with computing resources in the project. During the first months of operation this infrastructure has already been significantly exploited to deliver the first results towards the project's goals. The OpenNebula virtual management software has been used to install private clouds on different OS platforms. Two pre-production grid sites have been deployed in the private cloud and are used to test the implications of providing grid services in cloud environments. Experience from the deployment and operation of the above sites will help us identify the required tools and procedures for offering grid production sites over computing clouds. Finally the document presents related work, relevant to infrastructure operations, taking place in other projects and initiatives

Multi-core processors and the future of parallelism in software

The purpose of this thesis is to examine multi-core technology. Multi-core architecture provides benefits such as less power consumption, scalability, and improved application performance enabled by thread-level parallelism

Infrastructure Operations Final Report

This document serves as a final report of the activities and achievements of WP5 throughout the whole duration of the project. The document covers the areas of infrastructure operation, service provisioning, support, testing and benchmarking. In addition, the document provides a record of the practical knowledge accumulated during the provision of various public cloud services over a period of almost two years

Supercomputing Frontiers

This open access book constitutes the refereed proceedings of the 6th Asian Supercomputing Conference, SCFA 2020, which was planned to be held in February 2020, but unfortunately, the physical conference was cancelled due to the COVID-19 pandemic. The 8 full papers presented in this book were carefully reviewed and selected from 22 submissions. They cover a range of topics including file systems, memory hierarchy, HPC cloud platform, container image configuration workflow, large-scale applications, and scheduling

Neural Adaptive Admission Control Framework: SLA-driven action termination for real-time application service management

Although most modern cloud-based enterprise systems, or operating systems, do not commonly allow configurable/automatic termination of processes, tasks or actions, it is common practice for systems administrators to manually terminate, or stop, tasks or actions at any level of the system. The paper investigates the potential of automatic adaptive control with action termination as a method for adapting the system to more appropriate conditions in environments with established goals for both system’s performance and economics. A machine-learning driven control mechanism, employing neural networks, is derived and applied within data-intensive systems. Control policies that have been designed following this approach are evaluated under different load patterns and service level requirements. The experimental results demonstrate performance characteristics and benefits as well as implications of termination control when applied to different action types with distinct run-time characteristics. An automatic termination approach may be eminently suitable for systems with harsh execution time Service Level Agreements, or systems running under conditions of hard pressure on power supply or other constraints. The proposed control mechanisms can be combined with other available toolkits to support deployment of autonomous controllers in high-dimensional enterprise information systems

Evaluating the Impact of Security Measures on Performance of Secure Web Applications Hosted on Virtualised Platforms

The use of web applications has drastically increased over the years, and so has the need to secure these applications with effective security measures to ensure security and regulatory compliance. The problem arises when the impact and overheads associated with these security measures are not adequately quantified and factored into the design process of these applications. Organizations often resort to trading-off security compliance in order to achieve the required system performance. The aim of this research work is to quantify the impact of security measures on system performance of web applications and improve design decision-making in web application design process. This research work examines the implications of compliance and security measures on web applications and explores the possibility of extending the existing Queueing Network (QN) based models to predict the performance impact of security on web applications. The intention is that the results of this research work will assist system and web application designers in specifying adequate system capacity for secure web applications, hence ensuring acceptable system performance and security compliance. This research work comprises three quantitative studies organized in a sequential flow. The first study is an exploratory survey designed to understand the extent and importance of the security measures on system performance in organizations. The survey data was analyzed using descriptive statistics and Factor Analysis. The second study is an experimental study with a focus on causation. The study provided empirical data through sets of experiments proving the implications of security measures on a multi-tiered state-of-the-art web application - Microsoft SharePoint 2013. The experimental data were analyzed using the ANCOVA model. The third study is essentially a modeling-based study aimed at using the insights on the security implications provided by the second study. In the third study, using a well-established QN result - Mean Value Analysis (MVA) for closed networks, the study demonstrated how security measures could be incorporated into a QN model in an elegant manner with limited calculations. The results in this thesis indicated significant impact of security measures on web application with respect to response time, disk queue length, SQL latches and SQL database wait times. In a secure three-tiered web application the results indicated greater impacts on the web tier and database tier primarily due to encryption requirements dictated by several compliance standards, with smaller impact seen at the application tier. The modeling component of this thesis indicated a potential benefit in extending QN models to predict secure web application performance, although more work is needed to enhance the accuracy of the model. Overall, this research work contributes to professional practice by providing performance evaluation and predictive techniques for secure web applications that could be used in system design. From performance evaluations and QN modeling perspective, although three-tiered web application modeling has been widely studied, the view in this thesis is that this is the first attempt to look at security compliance in a three-tiered web application modeling on virtualized platforms

Asynchronous Teams and Tasks in a Message Passing Environment

As the discipline of scientific computing grows, so too does the "skills gap" between the increasingly complex scientific applications and the efficient algorithms required. Increasing demand for computational power on the march towards exascale requires innovative approaches. Closing the skills gap avoids the many pitfalls that lead to poor utilisation of resources and wasted investment. This thesis tackles two challenges: asynchronous algorithms for parallel computing and fault tolerance. First I present a novel asynchronous task invocation methodology for Discontinuous Galerkin codes called enclave tasking. The approach modifies the parallel ordering of tasks that allows for efficient scaling on dynamic meshes up to 756 cores. It ensures high levels of concurrency and intermixes tasks of different computational properties. Critical tasks along domain boundaries are prioritised for an overlap of computation and communication. The second contribution is the teaMPI library, forming teams of MPI processes exchanging consistency data through an asynchronous "heartbeat". In contrast to previous approaches, teaMPI operates fully asynchronously with reduced overhead. It is also capable of detecting individually slow or failing ranks and inconsistent data among replicas. Finally I provide an outlook into how asynchronous teams using enclave tasking can be combined into an advanced team-based diffusive load balancing scheme. Both concepts are integrated into and contribute towards the ExaHyPE project, a next generation code that solves hyperbolic equation systems on dynamically adaptive cartesian grids

Security Assessment of Select Computer Systems under Distributed Denial of Service Attacks Performed by Botnets

With the passage of time Internet connectivity has been found on more electronic devices than in previous decades. This continuous development may be perceived as a double-edged sword. While the world becomes more Internet-connected and quality of life increases, the lack of proper security protocols in some of these electronic devices permits illegitimate parties to take control of them for their own malicious purposes. This thesis studies the impact of compromised Internet-connected devices on commonly used server operating systems: Microsoft’s Windows Server 2012 R2 Datacenter, Windows Server 2016 Datacenter, and Apple’s macOS 10.13.6 High Sierra deployed on Apple’s Mac Pro Server (Mac Pro Mid2010). Their performance under prevalent Distributed Denial of Service Attacks will be evaluated at equal attack traffic loads while utilizing three networks of simulated botnets. While the server was under attack connection rate, network traffic, processor utilization, processor temperature, and memory utilization were evaluated for computer systems under consideration

Resource Management in Large-scale Systems

The focus of this thesis is resource management in large-scale systems. Our primary concerns are energy management and practical principles for self-organization and self-management. The main contributions of our work are: 1. Models. We proposed several models for different aspects of resource management, e.g., energy-aware load balancing and application scaling for the cloud ecosystem, hierarchical architecture model for self-organizing and self-manageable systems and a new cloud delivery model based on auction-driven self-organization approach. 2. Algorithms. We also proposed several different algorithms for the models described above. Algorithms such as coalition formation, combinatorial auctions and clustering algorithm for scale-free organizations of scale-free networks. 3. Evaluation. Eventually we conducted different evaluations for the proposed models and algorithms in order to verify them. All the simulations reported in this thesis had been carried out on different instances and services of Amazon Web Services (AWS). All of these modules will be discussed in detail in the following chapters respectively