Function-as-a-Service Benchmarking Framework

Cloud Service Providers deliver their products in form of ”as-a-Service”, which are typically categorized by the level of abstraction. This approach hides the implementation details and shows only functionality to the user. However, the problem is that it is hard to measure the performance of Cloud services, because they behave like black boxes. Especially with Function-as-a-Service it is even more difficult because it completely hides server and infrastructure management from users by design. Cloud Service Prodivers usually restrict the maximum size of code, memory and runtime of Cloud Functions. Nevertheless, users need clarification if more ressources are needed to deliver services in high quality. In this regard, we present the architectural design of a new Function-as-a-Service benchmarking tool, which allows users to evaluate the performance of Cloud Functions. Furthermore, the capabilities of the framework are tested on an isolated platform with a specific workload. The results show that users are able to get insights into Function-as-a-Service environments. This, in turn, allows users to identify factors which may slow down or speed up the performance of Cloud Functions

Adapting Microservices in the Cloud with FaaS

This project involves benchmarking, microservices and Function-as-a-service (FaaS) across the dimensions of performance and cost. In order to do a comparison this paper proposes a benchmark framework

Reference Models and Incentive Regulation of Electricity Distribution Networks: An Evaluation of Sweden’s Network Performance Assessment Model (NPAM)

The world-wide electricity sector reforms have led to a search for alternative and innovative approaches to regulation to promote efficiency improvement in the natural monopoly electricity networks. A number of countries have used incentive regulation models based on efficiency benchmarking of the electricity network utilities. While most regulators have opted adopted parametric and non-parametric frontier-based methods of benchmarking some have used engineering designed ‘reference firm’ or ‘norm’ models for the purpose. This paper examines the incentive properties and other related aspects of the norm model NPAM used in regulation of distribution networks in Sweden and compares these with those of frontier-based benchmarking methods. We identify a number of important differences between the two approaches to regulation benchmarking that are not readily apparent and discuss their ramifications for the regulatory objectives and process

A Role for Management Accountants in Best Practice Benchmarking

Best practice benchmarking (benchmarking for short) generally refers to the pursuit by organisations of enhanced performance by learning from the successful practices of others. Comparisons of processes which contribute to strategic success are made with other parts of the same organisation; competitors; or organisations operating comparable processes in a context which is in some way relevant. Benchmarking continues to grow in popularity in both private and public sector organisations – but does it always produce the desired outcomes? Although spectacular gains from benchmarking are claimed particularly in practitioner literature, there is also growing evidence of disappointment with the effectiveness of benchmarking. It can be very time consuming to undertake and manage, and ensuring that sharing information with competitors is to the mutual advantage of partner organisations is difficult. With this in mind, it is important to recognise that management accountants play pivotal roles at organisational interfaces and therefore could play a (more) significant part in successful benchmarking activities. This paper will report on an ongoing research project at the Open University Business School, funded by the Chartered Institute of Management Accountants, aimed at understanding, in depth, the processes which are undertaken by Management Accountants, in the name of benchmarking. The project team are using postal questionnaires and case studies to identify the features of successful benchmarking practice, and the characteristics of benchmarking organisations or benchmarking processes which are considered to be problematic. This research centres on an extensive survey of Management Accountants. This research has begun to identify the contribution which Management Accountants can make to successful benchmarking and the factors which have led organisations to abandon benchmarking activities. This study is also facilitating better understanding of the relationship between organisational size and level of benchmarking activity, the impact of benchmarking clubs, and the perceived costs and benefits of benchmarking to stakeholders. The final phase 3 of this research will focus on providing innovative ways to make the findings available to management accounting practitioners

Performance measurement in the service business: the facilities management function

The nature of performance measurement has changed over the past few decades. Generally performance measurement indicates successful management in the fulfilment of organisation goals. In service businesses measuring customer satisfaction and service quality has become an industry standard and as FM is becomes more widely recognised as a component in the business value chain and corporate strategic objectives, the adoption of performance indicators that relate directly to the core business driver is key to success. This paper examines the state of knowledge of performance measurement in a facilities management context, expounds and reveals the role that performance measurement plays in the overall efficiency of the FM service function in relation to service business operation. The paper suggests that a fully developed performance measurement solution can deliver as a business tool whilst acting as a driver in the innovation process of service organisation

Performance of Network and Service Monitoring Frameworks

The efficiency and the performance of anagement systems is becoming a hot research topic within the networks and services management community. This concern is due to the new challenges of large scale managed systems, where the management plane is integrated within the functional plane and where management activities have to carry accurate and up-to-date information. We defined a set of primary and secondary metrics to measure the performance of a management approach. Secondary metrics are derived from the primary ones and quantifies mainly the efficiency, the scalability and the impact of management activities. To validate our proposals, we have designed and developed a benchmarking platform dedicated to the measurement of the performance of a JMX manager-agent based management system. The second part of our work deals with the collection of measurement data sets from our JMX benchmarking platform. We mainly studied the effect of both load and the number of agents on the scalability, the impact of management activities on the user perceived performance of a managed server and the delays of JMX operations when carrying variables values. Our findings show that most of these delays follow a Weibull statistical distribution. We used this statistical model to study the behavior of a monitoring algorithm proposed in the literature, under heavy tail delays distribution. In this case, the view of the managed system on the manager side becomes noisy and out of date