A grid middleware for distributed Java computing with MPI binding and process migration supports

"Grid" computing has emerged as an important new research field. With years of efforts, grid researchers have successfully developed grid technologies including security solutions, resource management protocols, information query protocols, and data management services. However, as the ultimate goal of grid computing is to design an infrastructure which supports dynamic, cross-organizational resource sharing, there is a need of solutions for efficient and transparent task re-scheduling in the grid. In this research, a new grid middleware is proposed, called G-JavaMPI. This middleware adds the parallel computing capability of Java to the grid with the support of a Grid-enabled message passing interface (MPI) for inter-process communication between Java processes executed at different grid points. A special feature of the proposed G-JavaMPI is the support of Java process migration with post-migration message redirection. With these supports, it is possible to migrate executing Java process from site to site for continuous computation, if some site is scheduled to be turned down for system reconfiguration. Moreover, the proposed G-JavaMPI middleware is very portable since it requires no modification of underlying OS, Java virtual machine, and MPI package. Preliminary performance tests have been conducted. The proposed mechanisms have shown good migration efficiency in a simulated grid environment.postprin

Heterogeneous Strong Computation Migration

The continuous increase in performance requirements, for both scientific computation and industry, motivates the need of a powerful computing infrastructure. The Grid appeared as a solution for inexpensive execution of heavy applications in a parallel and distributed manner. It allows combining resources independently of their physical location and architecture to form a global resource pool available to all grid users. However, grid environments are highly unstable and unpredictable. Adaptability is a crucial issue in this context, in order to guarantee an appropriate quality of service to users. Migration is a technique frequently used for achieving adaptation. The objective of this report is to survey the problem of strong migration in heterogeneous environments like the grids', the related implementation issues and the current solutions.Comment: This is the pre-peer reviewed version of the following article: Milan\'es, A., Rodriguez, N. and Schulze, B. (2008), State of the art in heterogeneous strong migration of computations. Concurrency and Computation: Practice and Experience, 20: 1485-1508, which has been published in final form at http://onlinelibrary.wiley.com/doi/10.1002/cpe.1287/abstrac

An Optimizing Java Translation Framework for Automated Checkpointing and Strong Mobility

Long-running programs, e.g., in high-performance computing, need to write periodic checkpoints of their execution state to disk to allow them to recover from node failure. Manually adding checkpointing code to an application, however, is very tedious. The mechanisms needed for writing the execution state of a program to disk and restoring it are similar to those needed for migrating a running thread or a mobile object. We have extended a source-to-source translation scheme that allows the migration of mobile Java objects with running threads to make it more general and allow it to be used for automated checkpointing. Our translation scheme allows serializable threads to be written to disk or migrated with a mobile agent to a remote machine. The translator generates code that maintains a serializable run-time stack for each thread as a Java data structure. While this results in significant run-time overhead, it allows the checkpointing code to be generated automatically. We improved the locking mechanism that is needed to protect the run-time stack as well as the translation scheme. Our experimental results demonstrate an speedup of the generated code over the original translator and show that the approach is feasible in practice

Universal Mobile Service Execution Framework for Device-To-Device Collaborations

There are high demands of effective and high-performance of collaborations between mobile devices in the places where traditional Internet connections are unavailable, unreliable, or significantly overburdened, such as on a battlefield, disaster zones, isolated rural areas, or crowded public venues. To enable collaboration among the devices in opportunistic networks, code offloading and Remote Method Invocation are the two major mechanisms to ensure code portions of applications are successfully transmitted to and executed on the remote platforms. Although these domains are highly enjoyed in research for a decade, the limitations of multi-device connectivity, system error handling or cross platform compatibility prohibit these technologies from being broadly applied in the mobile industry. To address the above problems, we designed and developed UMSEF - an Universal Mobile Service Execution Framework, which is an innovative and radical approach for mobile computing in opportunistic networks. Our solution is built as a component-based mobile middleware architecture that is flexible and adaptive with multiple network topologies, tolerant for network errors and compatible for multiple platforms. We provided an effective algorithm to estimate the resource availability of a device for higher performance and energy consumption and a novel platform for mobile remote method invocation based on declarative annotations over multi-group device networks. The experiments in reality exposes our approach not only achieve the better performance and energy consumption, but can be extended to large-scaled ubiquitous or IoT systems

Supporting Context-Aware Application Development in Ad Hoc Mobile Networks

Some of the most dynamic systems being built today consist of physically mobile hosts and logically mobile agents. Such systems exhibit frequent configuration changes and a great deal of resource variability. Applications executing under these circumstances need to react continuously and rapidly to changes in operating conditions and must adapt their behavior accordingly. Applications with these capabilities are referred to as context-aware. Much of the current work on context-aware computing relies on information directly available to an application via context sensors on its local host, e.g., user profile, host location, time of day, resource availability, and quality of service measurements. The work reported in this dissertation starts by building a new perspective on context-awareness, in which the context includes, in principle, any information available in the ad hoc network but is restricted, in practice, to specific projections of the overall context. This work reports on the design and implementation of a middleware model that brings this notion of context to the application programmer. Another important aspect of the software engineering process is the ability to reason formally about the programs we create. This dissertation details initial steps to create formal reasoning mechanisms dedicated to the needs of context-aware applications. The results of this work simplify application development in ad hoc mobile networks from a design and implementation perspective and through formal reasoning

Flexible distributed computing with volunteered resources

PhDNowadays, computational grids have evolved to a stage where they can comprise many volunteered resources owned by different individual users and/or institutions, such as desktop grids and volunteered computing grids. This brings benefits for large-scale computing, as more resources are available to exploit. On the other hand, the inherent characteristics of the volunteered resources bring some challenges for efficiently exploiting them. For example, jobs may not be able to be executed by some resources, as the computing resources can be heterogeneous. Furthermore, the resources can be volatile as the resource owners usually have the right to decide when and how to donate the idle Central Processing Unit (CPU) cycles of their computers. Therefore, in order to utilise volunteered resources efficiently, this research investigated solutions from different aspects. Firstly, this research proposes a new computational Grid architecture based on Java and Java application migration technologies to provide fundamental support for coping with these challenges. This proposed architecture supports heterogeneous resources, ensuring local activities are not affected by Grid jobs and enabling resources to carry out live and automatic Java application migration. Secondly, this research work proposes some job-scheduling and migration algorithms based on resource availability prediction and/or artificial intelligence techniques. To examine the proposed algorithms, this work includes a series of experiments in both synthetic and practical scenarios and compares the performance of the proposed algorithms with existing ones across a variety of scenarios. According to the critical assessment, each algorithm has its own distinct advantages and performs well when certain conditions are met. In addition, this research analyses the characteristics of resources in terms of the availability pattern of practical volunteer-based grids. The analysis shows that each environment has its own characteristics and each volunteered resource’s availability tends to possess weak correlations across different days and times-of-day.British Telco

An Autonomic Cross-Platform Operating Environment for On-Demand Internet Computing

The Internet has evolved into a global and ubiquitous communication medium interconnecting powerful application servers, diverse desktop computers and mobile notebooks. Along with recent developments in computer technology, such as the convergence of computing and communication devices, the way how people use computers and the Internet has changed people´s working habits and has led to new application scenarios. On the one hand, pervasive computing, ubiquitous computing and nomadic computing become more and more important since different computing devices like PDAs and notebooks may be used concurrently and alternately, e.g. while the user is on the move. On the other hand, the ubiquitous availability and pervasive interconnection of computing systems have fostered various trends towards the dynamic utilization and spontaneous collaboration of available remote computing resources, which are addressed by approaches like utility computing, grid computing, cloud computing and public computing. From a general point of view, the common objective of this development is the use of Internet applications on demand, i.e. applications that are not installed in advance by a platform administrator but are dynamically deployed and run as they are requested by the application user. The heterogeneous and unmanaged nature of the Internet represents a major challenge for the on demand use of custom Internet applications across heterogeneous hardware platforms, operating systems and network environments. Promising remedies are autonomic computing systems that are supposed to maintain themselves without particular user or application intervention. In this thesis, an Autonomic Cross-Platform Operating Environment (ACOE) is presented that supports On Demand Internet Computing (ODIC), such as dynamic application composition and ad hoc execution migration. The approach is based on an integration middleware called crossware that does not replace existing middleware but operates as a self-managing mediator between diverse application requirements and heterogeneous platform configurations. A Java implementation of the Crossware Development Kit (XDK) is presented, followed by the description of the On Demand Internet Computing System (ODIX). The feasibility of the approach is shown by the implementation of an Internet Application Workbench, an Internet Application Factory and an Internet Peer Federation. They illustrate the use of ODIX to support local, remote and distributed ODIC, respectively. Finally, the suitability of the approach is discussed with respect to the support of ODIC

Improving efficiency, scalability and efficacy of adaptive computation offloading in pervasive computing environments

As computing becomes more mobile and pervasive, there is a growing demand for increasingly rich, and therefore more computationally heavy, applications to run in mobile spaces. However, there exists a disparity between mobile platforms and the desktop environments upon which computationally heavy applications have traditionally run, which is likely to persist as both domains evolve at a competing pace. Consequently, an active research area is Adaptive Computation Offloading or cyber foraging that dynamically distributes application functionality to available peer devices according to resource availability and application behaviour. Integral to any offloading strategy is an adaptive decision making algorithm that computes the optimal placement of application components to remote devices based on changing environmental context. As this decision is typically computed by constrained devices and may occur frequently in dynamic environments, such algorithms should be both resource efficient and yield efficacious adaptation results. However, existing adaptive offloading approaches incur a number of overheads, which limit their applicability in mobile and pervasive spaces. This thesis is concerned with improving upon these limitations by specifically focusing on the efficiency, scalability and efficacy aspects of two major sub processes of adaptation: 1) Adaptive Candidate Device Selection and 2) Adaptive Object Topology Computation. To this end, three novel approaches are proposed. Firstly, a distributed approach to candidate device selection, which reduces the need to communicate collaboration metrics, and allows for the partial distribution of adaptation decision-making, is proposed. The approach is shown to reduce network consumption by over 90% and power consumption by as much as 96%, while maintaining linear memory complexity in contrast to the quadratic complexity of an existing approach. Hence, the approach presents a more efficient and scalable alternative for candidate device selection in mobile and pervasive environments. Secondly, with regards to the efficacy of adaptive object topology computation, a new type of adaptation granularity that combines the efficacy of fine-grained adaptation with the efficiency of coarse level approaches is proposed. The approach is shown to improve the efficacy of adaptation decisions by reducing network overheads by a minimum of 17% to as much 99%, while maintaining comparable decision making efficiency to coarse level adaptation. Thirdly, with regards to efficiency and scalability of object topology computation, a novel distributed approach to computing adaptation decisions is proposed, in which each device maintains a distributed local application sub-graph, consisting only of components in its own memory space. The approach is shown to reduce network cost by 100%, collaboration-wide memory cost by between 37% and 50%, battery usage by between 63% and 93%, and adaptation time by between 19% and 98%. Lastly, since improving the utility of adaptation in mobile and pervasive environments requires the simultaneous improvement of its sub processes, an adaptation engine, which consolidates the individual approaches presented above, is proposed. The consolidated adaptation engine is shown to improve the overall efficiency, scalability and efficacy of adaptation under a varying range of environmental conditions, which simulate dynamic and heterogeneous mobile environments