Execution of unmodified applications on distributed storage and compute resources

Processing of unmodified applications on distributed storage and compute resources can be achieved by presenting the application with a virtual machine interface that abstracts access to the resources. This approach decouples the problem of mapping an application to available resources into two independent problems, (a) writing, compiling or translating the application to the virtual machine interface, and (b) building an efficient implementation of the virtual machine on top of the resources. The key challenges with this approach are: (a) how should the underlying resources be virtualized, and (b) at which layer of system software should the virtualization be implemented. The resolution of these two issues, depends on the characteristics of the applications to be supported, and the target distributed resources. In this thesis, the above virtualization approach is investigated in the context of three specific target systems. ‘Software-extended-machine-architecture’ is an abstract interface at the ISA-level, to exploit resources at intelligent peripherals within a node, that can be implemented via the compiler framework and application runtime system. The abstract ISA consists of the ISA of the main CPU extended by a soft ISA to program devices. SCODE, a proposal for a soft ISA that expresses the fine grained data-parallelism in general-purpose applications demonstrates the applicability of the proposed approach to exploit resources at intelligent streaming data devices within a node. The ‘delegated process’ abstraction at the OS-level, enables sequential distributed processing of multiprogramming workloads across closely coupled nodes managed as independent systems. An implementation in the Linux kernel demonstrates the viability of the approach by enabling systems that are out of memory to support memory-intensive workloads by borrowing memory and compute resources from remote machines. The ‘abstract job’ interface in the grid-middleware level, provides grid applications with a single system image to grid resources. Implementations in deployed grid-computing environments, PUNCH and In-VIGO, demonstrate the portability and scalability of the proposed interface by allowing the incorporation of new types of heterogeneous platforms with little effort and overhead. Successful application of the proposed virtualization approach in the above distinct scenarios provides evidence for its general applicability

Performance and Interoperability Issues in Incorporating Cluster Management Systems within a Wide-Area Network-Computing Environment

This paper describes the performance and interoperability issues that arise in the process of integrating cluster management systems into a wide-area networkcomputing environment, and provides solutions in the context of the Purdue University Network Computing Hubs (PUNCH). The described solution provides users with a single point of access to resources spread across administrative domains, and an intelligent translation process makes it possible for users to submit jobs to different types of cluster management systems in a transparent manner. The approach does not require any modifications to the cluster management software; however, call-back and caching capabilities that would improve performance and make such systems more interoperable with wide-area computing systems are discussed. 1. Introduction Cluster management systems manage access to workstations, servers, and specialized machines distributed across local-area networks. From a user's perspective, the systems provide a cent..

Security Implications of Making Computing Resources Available via Computational Grids

This report inrvestigates the issues of securing access to computin,g resources in computation. al grids. Grid en,viron,men.ts are built orb top of platforms that corrtrol access to resources within a sin,gle adenin.istrative domairr, at the g.r-an,ular.ityo f a use,r. In wide-area multi-domain. grid en.viron,men.ts, the overhead of mainiainin,g user accounts is prohibitive, a.rad secu7.in,g access to resources via user accountability is impractical. Typically, these issues are han,dled by inlplenlert,ting checks that guaran,tee the safety of applic~ations, so [hat they can rurt. in shared use^ accour1.t~. This work shows that safety checks - lan,guage-based, compile-time, link-time, load-lime - curren.tly implemen.ted in most grid en.virorrmen.ts are either in,adequute or limit allowed grid users and applicatiorrs. Techn.ique.s without such li7nitations are presen.ted. Shadow accoun,ts allow reu.se of user accounts without administrative overheads, and run-time solution~s - run-time 7non.itorirt.y and virtu.tr1 machines - allow arbilrary code to execute while erilforciny a given resource access policy

grids: the In-VIGO system

virtualized resources to virtual computin

A network-computing infrastructure for tool experimentation applied to computer architecture education

Computer architects increasingly depend on the use of software tools to evaluate and investigate the design of computer systems. It is therefore very important that educators in this field promote extensive tool-based experimentation by students in architecture classes. However, the integration of today's complex architecture tools into curricula poses several challenges to an instructor, including management of powerful computing resources, software installation and maintenance, and development of tool-specific educational material. This paper describes how these challenges are addressed by a universally accessible network-computing infrastructure --- NETCARE --- that provides educators with a Web portal to access computing resources, executable tools and educational material