Scalable Performance Analysis of Massively Parallel Stochastic Systems

The accurate performance analysis of large-scale computer and communication systems is directly inhibited by an exponential growth in the state-space of the underlying Markovian performance model. This is particularly true when considering massively-parallel architectures such as cloud or grid computing infrastructures. Nevertheless, an ability to extract quantitative performance measures such as passage-time distributions from performance models of these systems is critical for providers of these services. Indeed, without such an ability, they remain unable to offer realistic end-to-end service level agreements (SLAs) which they can have any confidence of honouring. Additionally, this must be possible in a short enough period of time to allow many different parameter combinations in a complex system to be tested. If we can achieve this rapid performance analysis goal, it will enable service providers and engineers to determine the cost-optimal behaviour which satisfies the SLAs. In this thesis, we develop a scalable performance analysis framework for the grouped PEPA stochastic process algebra. Our approach is based on the approximation of key model quantities such as means and variances by tractable systems of ordinary differential equations (ODEs). Crucially, the size of these systems of ODEs is independent of the number of interacting entities within the model, making these analysis techniques extremely scalable. The reliability of our approach is directly supported by convergence results and, in some cases, explicit error bounds. We focus on extracting passage-time measures from performance models since these are very commonly the language in which a service level agreement is phrased. We design scalable analysis techniques which can handle passages defined both in terms of entire component populations as well as individual or tagged members of a large population. A precise and straightforward specification of a passage-time service level agreement is as important to the performance engineering process as its evaluation. This is especially true of large and complex models of industrial-scale systems. To address this, we introduce the unified stochastic probe framework. Unified stochastic probes are used to generate a model augmentation which exposes explicitly the SLA measure of interest to the analysis toolkit. In this thesis, we deploy these probes to define many detailed and derived performance measures that can be automatically and directly analysed using rapid ODE techniques. In this way, we tackle applicable problems at many levels of the performance engineering process: from specification and model representation to efficient and scalable analysis

Stochastic scheduling and workload allocation : QoS support and profitable brokering in computing grids

Abstract: The Grid can be seen as a collection of services each of which performs some functionality. Users of the Grid seek to use combinations of these services to perform the overall task they need to achieve. In general this can be seen as aset of services with a workflow document describing how these services should be combined. The user may also have certain constraints on the workflow operations, such as execution time or cost ----t~ th~ user, specified in the form of a Quality of Service (QoS) document. The users . submit their workflow to a brokering service along with the QoS document. The brokering service's task is to map any given workflow to a subset of the Grid services taking the QoS and state of the Grid into account -- service availability and performance. We propose an approach for generating constraint equations describing the workflow, the QoS requirements and the state of the Grid. This set of equations may be solved using Mixed-Integer Linear Programming (MILP), which is the traditional method. We further develop a novel 2-stage stochastic MILP which is capable of dealing with the volatile nature of the Grid and adapting the selection of the services during the lifetime of the workflow. We present experimental results comparing our approaches, showing that the . 2-stage stochastic programming approach performs consistently better than other traditional approaches. Next we addresses workload allocation techniques for Grid workflows in a multi-cluster Grid We model individual clusters as MIMIk. queues and obtain a numerical solutio~ for missed deadlines (failures) of tasks of Grid workflows. We also present an efficient algorithm for obtaining workload allocations of clusters. Next we model individual cluster resources as G/G/l queues and solve an optimisation problem that minimises QoS requirement violation, provides QoS guarantee and outperforms reservation based scheduling algorithms. Both approaches are evaluated through an experimental simulation and the results confirm that the proposed workload allocation strategies combined with traditional scheduling algorithms performs considerably better in terms of satisfying QoS requirements of Grid workflows than scheduling algorithms that don't employ such workload allocation techniques. Next we develop a novel method for Grid brokers that aims at maximising profit whilst satisfying end-user needs with a sufficient guarantee in a volatile utility Grid. We develop a develop a 2-stage stochastic MILP which is capable of dealing with the volatile nature . of the Grid and obtaining cost bounds that ensure that end-user cost is minimised or satisfied and broker's profit is maximised with sufficient guarantee. These bounds help brokers know beforehand whether the budget limits of end-users can be satisfied and. if not then???????? obtain appropriate future leases from service providers. Experimental results confirm the efficacy of our approach.Imperial Users onl

Timed runtime monitoring for multiparty conversations

We propose a dynamic verification framework for protocols in real-time distributed systems. The framework is based on Scribble, a tool-chain for design and verification of choreographies based on multiparty session types, which we have developed with our industrial partners. Drawing from recent work on multiparty session types for real-time interactions, we extend Scribble with clocks, resets, and clock predicates in order to constrain the times inwhich interactions occur.We present a timedAPI for Python to programdistributed implementations of Scribble specifications. A dynamic verification framework ensures the safe execution of applications written with our timed API: we have implemented dedicated runtime monitors that check that each interaction occurs at a correct timing with respect to the corresponding Scribble specification. To demonstrate the practicality of the proposed framework, we express and verify four categories of widely used temporal patterns from use cases in literature.We analyse the performance of our implementation via benchmarking and show negligible overhead

Estimation of the statistics of rare events in data communications systems

There are many examples of rare events in telecommunications systems, including buffer overflows in queueing systems, cycle slipping in phase-locked loops and the escape of adaptive equalizers from local (possibly incorrect) equilibria. In many cases, factors such as the high cost of the occurrence of these events mean that their statistics are of interest in spite of their rarity. The estimation of the statistics of these rare events via direct simulation is very time consuming, simply because of their rarity. In fact, the required simulation time can be so high as to make simulation not just difficult, but impossible. One technique that can be used to speed up simulations of rare events is importance sampling, in which the statistics of the event in which we are interested are inferred from the statistics (obtained by simulation) of some (less rare) event in a different system. Because the events are less rare, the simulation time is reduced. However, there remains the problem of maximizing the speedup to ensure that the simulation time is minimized. It has been shown previously that as the rarity of the events increases, large deviations theory can be used to create a simulation system that is optimal in the sense of minimizing the variance of a probability estimator in the simulation of a rare event. In this thesis, we extend these results, and also apply them to a number of specific applications for which we obtain analytic expressions for an asymptotically optimal simulation system. Examples studied include multiple-priority data streams and a number of queues with deterministic servers, which can be used in the modeling of asynchronous transfer mode (ATM) switches. In the case of buffer overflows in queueing systems, it will be shown that the required simulation time is reduced from being exponential in the buffer size for direct simulation , to being linear in the buffer size using the asymptotically optimal simulation system, and that this holds even for relatively small buffer sizes. While much of the previous work on fast simulation of rare events has concentrated on the use of large deviations and expon-ential changes of measure, we look beyond this class, and show that it is possible to obtain larger increases in simulation speed, using, for example, the reverse-time model of the system being studied. In fact , it is possible to obtain an infinite speedup. However, doing this may require omniscience, i.e. effectively knowing the answer before we start. In addition to the investigation of methods for performing fast simulation, the relationship between optimal control, large deviations and reverse-time modeling is explored, with particular reference to rare events. It is shown that, in addition to the previously known relationship between optimal control and large deviations, a similar relationship exists between optimal control and reverse-time modeling, in which the trajectory defining the solution of the optimal control problem in which control energy is minimized defines the mean path of the reverse-time model of the process

Computing optical flow using fast total variation

During my internship, I was in charge of implementing a GPU version of the optical flow algorithm. The optical flow algorithm is based on the total variation features described in my bibliography

10381 Summary and Abstracts Collection -- Robust Query Processing

Dagstuhl seminar 10381 on robust query processing (held 19.09.10 - 24.09.10) brought together a diverse set of researchers and practitioners with a broad range of expertise for the purpose of fostering discussion and collaboration regarding causes, opportunities, and solutions for achieving robust query processing. The seminar strove to build a unified view across the loosely-coupled system components responsible for the various stages of database query processing. Participants were chosen for their experience with database query processing and, where possible, their prior work in academic research or in product development towards robustness in database query processing. In order to pave the way to motivate, measure, and protect future advances in robust query processing, seminar 10381 focused on developing tests for measuring the robustness of query processing. In these proceedings, we first review the seminar topics, goals, and results, then present abstracts or notes of some of the seminar break-out sessions. We also include, as an appendix, the robust query processing reading list that was collected and distributed to participants before the seminar began, as well as summaries of a few of those papers that were contributed by some participants

Proceedings of the 8th Cologne-Twente Workshop on Graphs and Combinatorial Optimization

International audienceThe Cologne-Twente Workshop (CTW) on Graphs and Combinatorial Optimization started off as a series of workshops organized bi-annually by either Köln University or Twente University. As its importance grew over time, it re-centered its geographical focus by including northern Italy (CTW04 in Menaggio, on the lake Como and CTW08 in Gargnano, on the Garda lake). This year, CTW (in its eighth edition) will be staged in France for the first time: more precisely in the heart of Paris, at the Conservatoire National d’Arts et Métiers (CNAM), between 2nd and 4th June 2009, by a mixed organizing committee with members from LIX, Ecole Polytechnique and CEDRIC, CNAM

Survey on time-delay approach to networked control

This paper provides a survey on time-delay approach to networked control systems (NCSs). The survey begins from a brief summary on fundamental network-induced issues in NCSs and the main approaches to the modelling of NCSs. In particular, a comprehensive introduction to time-delay approach to sampled-data and networked control is provided. Then, recent results on time-delay approach to event-triggered control are recalled. The survey highlights time-delay approach developed to modelling, analysis and synthesis of NCSs, under communication constraints, with a particular focus on Round-Robin, Try-once-discard and stochastic protocols. The time-delay approach allows communication delays to be larger than the sampling intervals in the presence of scheduling protocols. Moreover, some results on networked control of distributed parameter systems are surveyed. Finally, conclusions and some future research directions are briefly addressed