Distributed Management of Massive Data: an Efficient Fine-Grain Data Access Scheme

This paper addresses the problem of efficiently storing and accessing massive data blocks in a large-scale distributed environment, while providing efficient fine-grain access to data subsets. This issue is crucial in the context of applications in the field of databases, data mining and multimedia. We propose a data sharing service based on distributed, RAM-based storage of data, while leveraging a DHT-based, natively parallel metadata management scheme. As opposed to the most commonly used grid storage infrastructures that provide mechanisms for explicit data localization and transfer, we provide a transparent access model, where data are accessed through global identifiers. Our proposal has been validated through a prototype implementation whose preliminary evaluation provides promising results

Higher-order compatible finite element schemes for the nonlinear rotating shallow water equations on the sphere

We describe a compatible finite element discretisation for the shallow water equations on the rotating sphere, concentrating on integrating consistent upwind stabilisation into the framework. Although the prognostic variables are velocity and layer depth, the discretisation has a diagnostic potential vorticity that satisfies a stable upwinded advection equation through a Taylor-Galerkin scheme; this provides a mechanism for dissipating enstrophy at the gridscale whilst retaining optimal order consistency. We also use upwind discontinuous Galerkin schemes for the transport of layer depth. These transport schemes are incorporated into a semi-implicit formulation that is facilitated by a hybridisation method for solving the resulting mixed Helmholtz equation. We illustrate our discretisation with some standard rotating sphere test problems.Comment: accepted versio

Data Mining to Uncover Heterogeneous Water Use Behaviors From Smart Meter Data

Knowledge on the determinants and patterns of water demand for different consumers supports the design of customized demand management strategies. Smart meters coupled with big data analytics tools create a unique opportunity to support such strategies. Yet, at present, the information content of smart meter data is not fully mined and usually needs to be complemented with water fixture inventory and survey data to achieve detailed customer segmentation based on end use water usage. In this paper, we developed a data‐driven approach that extracts information on heterogeneous water end use routines, main end use components, and temporal characteristics, only via data mining existing smart meter readings at the scale of individual households. We tested our approach on data from 327 households in Australia, each monitored with smart meters logging water use readings every 5 s. As part of the approach, we first disaggregated the household‐level water use time series into different end uses via Autoflow. We then adapted a customer segmentation based on eigenbehavior analysis to discriminate among heterogeneous water end use routines and identify clusters of consumers presenting similar routines. Results revealed three main water end use profile clusters, each characterized by a primary end use: shower, clothes washing, and irrigation. Time‐of‐use and intensity‐of‐use differences exist within each class, as well as different characteristics of regularity and periodicity over time. Our customer segmentation analysis approach provides utilities with a concise snapshot of recurrent water use routines from smart meter data and can be used to support customized demand management strategies.TU Berlin, Open-Access-Mittel - 201