Analyzing the Combined Effects of Measurement Error and Perturbation Error on Performance Measurement

Dynamic performance analysis of executing programs commonly relies on statistical profiling techniques to provide performance measurement results. When a program execution is sampled we learn something about the examined program, but also change, to some extent, the program's interaction with the underlying system and thus its behavior. The amount we learn diminishes (statistically) with each sample taken, while the change we affect with the intrusive sampling risks growing larger. Effectively sampling programs is challenging largely because of the opposing effects of the decreasing sampling error and increasing perturbation error. Achieving the highest overall level of confidence in measurement results requires striking an appropriate balance between the tensions inherent in these two types of errors. Despite the popularity of statistical profiling, published material typically only explains in general qualitative terms the motivation of the systematic sampling rates used. Given the importance of sampling, we argue in favor of the general principle of deliberate sample size selection and have developed and tested a technique for doing so. We present our idea of sample rate selection based on abstract and mathematical performance measurement models we developed that incorporate the effect of sampling on both measurement accuracy and perturbation effects. Our mathematical model predicts the sampling size at which the combination of the residual measurement error and the accumulating perturbation error is minimized. Our evaluation of the model with simulation, calibration programs, and selected programs from the SPEC CPU 2006 and SPEC OMP 2001 benchmark suites indicates that this idea has promise. Our results show that the predicted sample size is generally close to the best sampling rate and effectively avoids bad choices. Most importantly, adaptive sample rate selection is shown to perform better than a single selected rate in most cases

Dataflow development of medium-grained parallel software

PhD ThesisIn the 1980s, multiple-processor computers (multiprocessors) based on conven- tional processing elements emerged as a popular solution to the continuing demand for ever-greater computing power. These machines offer a general-purpose parallel processing platform on which the size of program units which can be efficiently executed in parallel - the "grain size" - is smaller than that offered by distributed computing environments, though greater than that of some more specialised architectures. However, programming to exploit this medium-grained parallelism remains difficult. Concurrent execution is inherently complex, yet there is a lack of programming tools to support parallel programming activities such as program design, implementation, debugging, performance tuning and so on. In helping to manage complexity in sequential programming, visual tools have often been used to great effect, which suggests one approach towards the goal of making parallel programming less difficult. This thesis examines the possibilities which the dataflow paradigm has to offer as the basis for a set of visual parallel programming tools, and presents a dataflow notation designed as a framework for medium-grained parallel programming. The implementation of this notation as a programming language is discussed, and its suitability for the medium-grained level is examinedScience and Engineering Research Council of Great Britain EC ERASMUS schem

The exploitation of parallelism on shared memory multiprocessors

PhD ThesisWith the arrival of many general purpose shared memory multiple processor (multiprocessor) computers into the commercial arena during the mid-1980's, a rift has opened between the raw processing power offered by the emerging hardware and the relative inability of its operating software to effectively deliver this power to potential users. This rift stems from the fact that, currently, no computational model with the capability to elegantly express parallel activity is mature enough to be universally accepted, and used as the basis for programming languages to exploit the parallelism that multiprocessors offer. To add to this, there is a lack of software tools to assist programmers in the processes of designing and debugging parallel programs. Although much research has been done in the field of programming languages, no undisputed candidate for the most appropriate language for programming shared memory multiprocessors has yet been found. This thesis examines why this state of affairs has arisen and proposes programming language constructs, together with a programming methodology and environment, to close the ever widening hardware to software gap. The novel programming constructs described in this thesis are intended for use in imperative languages even though they make use of the synchronisation inherent in the dataflow model by using the semantics of single assignment when operating on shared data, so giving rise to the term shared values. As there are several distinct parallel programming paradigms, matching flavours of shared value are developed to permit the concise expression of these paradigms.The Science and Engineering Research Council

A testbed for embedded systems

Testing and Debugging are often the most difficult phase of software development. This is especially true of embedded systems which are usually concurrent, have real-time performance and correctness constraints and which execute in the field in an environment which may not permit internal scrutiny of the software behaviour. Although good software engineering practices help, they will never eliminate the need for testing and debugging. This is because failings in the specification and design are often only discovered through testing and understanding these failings and how to correct them comes from debugging. These observations suggest that embedded software should be designed in a way which makes testing and debugging easier and that tools which support these activities are required. Due to the often hostile environment in which the finished embedded system will function, it is necessary to have a platform which allows the software to be developed and tested "in vitro". The Testbed system achieves these goals by providing dynamic modification and process migration facilities for use during development as well as powerful monitoring and background debugging support. These facilities are built on a basic run-time harness supporting an event-driven programming model with a global communication mechanism. This programming model is well suited to the reactive nature of embedded systems. The main research contributions of this work are in the areas of finding deadlock-free, path-optimal routings for networks and of dynamic modification with automated conversion of data which may include pointers

The interaction network : a performance measurement and evaluation tool for loosely-coupled distributed systems

Much of today's computing is done on loosely-coupled distributed systems. Performance issues for such systems usually involve interactive performance, that is, system responsiveness as perceived by the user. The goal of the work described in this thesis has been to develop and implement tools and techniques for the measurement and evaluation of interactive performance in loosely-coupled distributed systems. The author has developed the concept of the interaction network, an acyclic directed graph designed to represent the processing performed by a distributed system in response to a user input. The definition of an interaction network is based on a general model of a loosely-coupled distributed system and a general model of user interactions. The author shows that his distributed system model is a valid abstraction for a wide range of present-day systems. Performance monitors for traditional time-sharing systems reported performance information, such as overall resource utilisations and queue lengths, for the system as a whole. Performance problems are now much more difficult, because systems are much more complex. Recent monitors designed specifically for distributed systems have tended to present performance information for execution of a distributed program, for example the time spent in each of a program's procedures. In the work described in this thesis, performance information is reported for one or more user interactions, where a user interaction is defined to be a single user input and all of the processing performed by the system on receiving that input. A user interaction is seen as quite different from a program execution; a user interaction includes the partial or total execution of one or more programs, and a program execution performs work as part of one or more user interactions. Several methods are then developed to show how performance information can be obtained from analysis of interaction networks. One valuable type of performance information is a decomposition of response time into times spent in each of some set of states, where each state might be defined in terms of the hardware and software resources used. Other performance information can be found from displays of interaction networks. The critical path through an interaction network is then defined as showing the set of activities such that at least one must be reduced in length if the response time of the interaction is to be reduced; the critical path is used in both response time decompositions and in displays of interaction networks. It was thought essential to demonstrate that interaction networks could be recorded for a working operating system. INMON, a prototype monitor based on the interaction network concept, has been constructed to operate in the SunOS environment. INMON consists of data collection and data analysis components. The data collection component, for example, involved the adding of 53 probes to the SunOS operating system kernel. To record interaction networks, a high-resolution global timebase is needed. A clock synchronisation program has been written to provide INMON with such a timebase. It is suggested that the method incorporates a number of improvements over other clock synchronisation methods. Several experiments have been performed to show that INMON can produce very detailed performance information for both individual user interactions and groups of user interactions, with user input being made through either character-based or graphical interfaces. The main conclusion reached in this thesis is that representing the processing component of a user interaction in an interaction network is a very valuable way of approaching the problem of measuring interactive performance in a loosely-coupled distributed system. An interaction network contains a very detailed record of the execution of an interaction and, from this record, a great deal of performance (and other) information can be derived. Construction of INMON has demonstrated that interaction networks can be identified, recorded, and analysed