Memory-aware sizing for in-memory databases

In-memory database systems are among the technological drivers of big data processing. In this paper we apply analytical modeling to enable efficient sizing of in-memory databases. We present novel response time approximations under online analytical processing workloads to model thread-level forkjoin and per-class memory occupation.We combine these approximations with a non-linear optimization program to minimize memory swapping in in-memory database clusters. We compare our approach with state-of-the-art response time approximations and trace-driven simulation using real data from an SAP HANA in-memory system and show that our optimization model is significantly more accurate than existing approaches at similar computational costs

A Bayesian Approach to Parameter Inference in Queueing Networks

The application of queueing network models to real-world applications often involves the task of estimating the service demand placed by requests at queueing nodes. In this article, we propose a methodology to estimate service demands in closed multiclass queueing networks based on Gibbs sampling. Our methodology requires measurements of the number of jobs at resources and can accept prior probabilities on the demands. Gibbs sampling is challenging to apply to estimation problems for queueing networks since it requires one to efficiently evaluate a likelihood function on the measured data. This likelihood function depends on the equilibrium solution of the network, which is difficult to compute in closed models due to the presence of the normalizing constant of the equilibrium state probabilities. To tackle this obstacle, we define a novel iterative approximation of the normalizing constant and show the improved accuracy of this approach, compared to existing methods, for use in conjunction with Gibbs sampling. We also demonstrate that, as a demand estimation tool, Gibbs sampling outperforms other popular Markov Chain Monte Carlo approximations. Experimental validation based on traces from a cloud application demonstrates the effectiveness of Gibbs sampling for service demand estimation in real-world studies

Recent developments in the data assimilation of AROME/HU numerical weather prediction model

A local three-dimensional variational data assimilation (DA) system was implemented operationally in AROME/HU (Application of Research to Operations at Mesoscale) non-hydrostatic mesoscale model at the Hungarian Meteorological Service (OMSZ) in 2013. In the first version, rapid update cycling (RUC) approach was employed with 3-hour frequency in local upper-air DA using conventional observations only. Optimal interpolation method was adopted for the surface data assimilation later in 2016. This paper describes the current developments showing the impact of more conventional and remote-sensing observations assimilated in this system, which reveals the benefit of additional local high-resolution observations. Furthermore, it is shown that an hourly assimilation-forecast cycle outperforms the 3-hourly updated system in our DA. Besides the upper-air assimilation developments, a simplified extended Kalman filter (SEKF) was also tested for surface data assimilation, showing promising performance on both the analyses and the forecasts of AROME/HU system

Model-Driven System Capacity Planning under Workload Burstiness

In this paper, we define and study a new class of capacity planning models called MAP queueing networks. MAP queueing networks provide the first analytical methodology to describe and predict accurately the performance of complex systems operating under bursty workloads, such as multi-tier architectures or storage arrays. Burstiness is a feature that significantly degrades system performance and that cannot be captured explicitly by existing capacity planning models. MAP queueing networks address this limitation by describing computer systems as closed networks of servers whose service times are Markovian Arrival Processes (MAPs), a class of Markov-modulated point processes that can model general distributions and burstiness. In this paper, we show that MAP queueing networks provide reliable performance predictions even if the service processes are bursty. We propose a methodology to solve MAP queueing networks by two state space transformations, which we call Linear Reduction (LR) and Quadratic Reduction (QR). These transformations dramatically decrease the number of states in the underlying Markov chain of the queueing network model. From these reduced state spaces, we obtain two classes of bounds on arbitrary performance indexes, e.g., throughput, response time, utilizations. Numerical experiments show that LR an QR bounds achieve good accuracy. We also illustrate the high effectiveness of the LR and QR bounds in the performance analysis of a real multi-tier architecture subject to TPC-W workloads that are characterized as bursty. These results promote MAP queueing networks as a new robust class of capacity planning models

LoPC-- modeling contention in parallel algorithms

Thesis (M.S.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1998.Includes bibliographical references (p. 43-44).by Matthew Frank.M.S

Adaptive governance of utilities: case of the water sector in an emerging market context

Adequate, equitable provision of essential resources requires governance that can adapt to the needs of a complex resource regime. Insufficient coordination and cooperation are barriers to governance of a resource system that is characterised by human and social interaction. This thesis explores how the application of governance frameworks for complex resource regimes, adaptive governance and social contracts that enable a diversity of perspectives on governance to inform understanding of cooperation in the provision of essential resources. Utilising an in-depth case study of water and sanitation provision in Medellin, Colombia, the thesis identifies insights from adaptive governance for the provision of essential resources through data-driven and theory-driven analytical approaches to: 1) test whether the system of water governance in Medellin is adaptive 2) describe the regime characteristics in comparison with existing theory on adaptive governance and assess alternative governing arrangements and 3) assess the social contracts within these governance arrangements. The results of semi-structured interviews with 30+ representatives from 6 stakeholder groups (utility provider, metropolitan authority, municipal authority, universities, community-based organisations and water user associations) indicate that the system of water governance in Medellin has: 1) adaptive governance in the policy domain and mechanisms for multi-stakeholder participation, 2) Strong features of polycentric governance associated with ‘bridging actors’, 3) Strong forms of monocentric governance among environmental and municipal authorities and 4) top-down, mixed and bottom-up social contract arrangements. These findings suggest a form of governance that is consistent with “malleable” governance the capacity of actors within a system to demonstrate different types of arrangements that evolve in relation to needs within the system. Contributions include a multi-disciplinary approach for navigating complex resource regimes and findings that provide a case study narrative of governance that moves towards malleability

Estimating multiclass service demand distributions using Markovian arrival processes

Building performance models for software services in DevOps is costly and error-prone. Accurate service demand distribution estimation is critical to precisely modeling queueing behaviors and performance prediction. However, current estimation methods focus on capturing the mean service demand, disregarding higher-order moments of the distribution that still can largely affect prediction accuracy. To address this limitation, we propose to estimate higher moments of the service demand distribution for a microservice from monitoring traces. We first generate a closed queueing model to abstract software performance and use it to model the departure process of requests completed by the software service as a Markovian arrival process. This allows formulating the estimation of service demand into an optimization problem, which aims to find the first multiple moments of the service demand distribution that maximize the likelihood of the MAP using generated the measured inter-departure times. We then estimate the service demand distribution for different classes of service with a maximum likelihood algorithm and novel heuristics to mitigate the computational cost of the optimization process for scalability. We apply our method to real traces from a microservice-based application and demonstrate that its estimations lead to greater prediction accuracy than exponential distributions assumed in traditional service demand estimation approaches for software services

The Journal of ERW and Mine Action Issue 14.3 (2010)

Focus: Looking Beyond Mine Action | Feature: Development and Funding | Special Report: Update on National Programs | Notes from the Field | Research & Developmen