Incremental View Maintenance For Collection Programming

In the context of incremental view maintenance (IVM), delta query derivation is an essential technique for speeding up the processing of large, dynamic datasets. The goal is to generate delta queries that, given a small change in the input, can update the materialized view more efficiently than via recomputation. In this work we propose the first solution for the efficient incrementalization of positive nested relational calculus (NRC+) on bags (with integer multiplicities). More precisely, we model the cost of NRC+ operators and classify queries as efficiently incrementalizable if their delta has a strictly lower cost than full re-evaluation. Then, we identify IncNRC+; a large fragment of NRC+ that is efficiently incrementalizable and we provide a semantics-preserving translation that takes any NRC+ query to a collection of IncNRC+ queries. Furthermore, we prove that incremental maintenance for NRC+ is within the complexity class NC0 and we showcase how recursive IVM, a technique that has provided significant speedups over traditional IVM in the case of flat queries [25], can also be applied to IncNRC+.Comment: 24 pages (12 pages plus appendix

Distributed k-core view materialization and maintenance for large dynamic graphs

Cataloged from PDF version of article.In graph theory, k-core is a key metric used to identify subgraphs of high cohesion, also known as the ‘dense’ regions of a graph. As the real world graphs such as social network graphs grow in size, the contents get richer and the topologies change dynamically, we are challenged not only to materialize k-core subgraphs for one time but also to maintain them in order to keep up with continuous updates. Adding to the challenge is that real world data sets are outgrowing the capacity of a single server and its main memory. These challenges inspired us to propose a new set of distributed algorithms for k-core view construction and maintenance on a horizontally scaling storage and computing platform. Our algorithms execute against the partitioned graph data in parallel and take advantage of k-core properties to aggressively prune unnecessary computation. Experimental evaluation results demonstrated orders of magnitude speedup and advantages of maintaining k-core incrementally and in batch windows over complete reconstruction. Our algorithms thus enable practitioners to create and maintain many k-core views on different topics in rich social network content simultaneously

Intelligent integrated maintenance for wind power generation

A novel architecture and system for the provision of Reliability Centred Maintenance (RCM) for offshore wind power generation is presented. The architecture was developed by conducting a bottom-up analysis of the data required to support RCM within this specific industry, combined with a top-down analysis of the required maintenance functionality. The architecture and system consists of three integrated modules for Intelligent Condition Monitoring, Reliability and Maintenance Modelling, and Maintenance Scheduling that provide a scalable solution for performing dynamic, efficient and cost effective preventative maintenance management within this extremely demanding renewable energy generation sector. The system demonstrates for the first time, the integration of state-of-the-art advanced mathematical techniques: Random Forests, Dynamic Bayesian Networks, and Memetic Algorithms in the development of an intelligent autonomous solution. The results from the application of the intelligent integrated system illustrated the automated detection of faults within a wind farm consisting of over 100 turbines, the modelling and updating of the turbines’ survivability and creation of a hierarchy of maintenance actions, and the optimising of the maintenance schedule with a view to maximising the availability and revenue generation of the turbines

Dynamic Planning of Vertical Road Signs Maintenance: Operational Methodology

This article is the first of two papers on a topic gaining ground in traffic and transport engineering, namely Dynamic Planning of Vertical Road Signs Maintenance. In particular, this first paper describes the methodologies used in determining the current state, the position, the type, the state of conservation and the efficiency of vertical road signs in the city of Forte dei Marmi (Tuscany, Italy). The ultimate goal is to set up a dynamic GIS-based road vertical signs archive and subsequently develop a forecast model. Such an operational procedure can be seen as a useful tool for setting up a Dynamic Maintenance Plan for all vertical signs along the roads of the municipal area. The methodology followed is the following: once the area of interest has been defined, the retroreflection parameter of the 752 vertical signs present in the area was measured using a retroreflectometer, recording the georeferenced data in a specific GIS database. This hands-on approach allows for both adding and removing a given sign, as well as sorting all monitored signs by age and efficiency in a dynamic operational maintenance view. Hence, the development of the forecasting model has been done, however, for this part reference should be made to the second paper. In the final part of this article, the concluding chapter relating to the creation of a dynamic maintenance plan and the conclusions on the methodology used and on possible ideas for further research developments have been included

Defining Epidermal Basal Cell States during Skin Homeostasis and Wound Healing Using Single-Cell Transcriptomics.

Our knowledge of transcriptional heterogeneities in epithelial stem and progenitor cell compartments is limited. Epidermal basal cells sustain cutaneous tissue maintenance and drive wound healing. Previous studies have probed basal cell heterogeneity in stem and progenitor potential, but a comprehensive dissection of basal cell dynamics during differentiation is lacking. Using single-cell RNA sequencing coupled with RNAScope and fluorescence lifetime imaging, we identify three non-proliferative and one proliferative basal cell state in homeostatic skin that differ in metabolic preference and become spatially partitioned during wound re-epithelialization. Pseudotemporal trajectory and RNA velocity analyses predict a quasi-linear differentiation hierarchy where basal cells progress from Col17a1Hi/Trp63Hi state to early-response state, proliferate at the juncture of these two states, or become growth arrested before differentiating into spinous cells. Wound healing induces plasticity manifested by dynamic basal-spinous interconversions at multiple basal transcriptional states. Our study provides a systematic view of epidermal cellular dynamics, supporting a revised "hierarchical-lineage" model of homeostasis

Adapting Materialized Views after Redefinitions: Techniques and a Performance Study

We consider a variant of the view maintenance problem: How does one keep a materialized view up-to-date when the view definition itself changes? Can one do better than recomputing the view from the base relations? Traditional view maintenance tries to maintain the materialized view in response to modifications to the base relations; we try to ``adapt'' the view in response to changes in the view definition. Such techniques are needed for applications where the user can change queries dynamically and see the changes in the results fast. Data archaeology, data visualization, and dynamic queries are examples of such applications. We consider all possible redefinitions of SQL\Select-\From-\Where-\Groupby-\Having, \Union, and \Minus\ views, and show how these views can be adapted using the old materialization for the cases where it is possible to do so. We identify extra information that can be kept with a materialization to facilitate redefinition.Multiple simultaneous changes to a view can be handled without necessarily materializing intermediate results. We identify guidelines for users and database administrators that can be used to facilitate efficient view adaptation. We perform a systematic experimental evaluation of our proposed techniques. Our evaluation indicates that adaptation is more efficient than rematerialization in most cases. Certain adaptation techniques can be up to1,000 times better. We also point out the physical layouts that can benefit adaptation

The countryside in urbanized Flanders: towards a flexible definition for a dynamic policy

The countryside, the rural area, the open space, … many definitions are used for rural Flanders. Everyone makes its own interpretation of the countryside, considering it as a place for living, working or recreating. The countryside is more than just a geographical area: it is an aggregate of physical, social, economic and cultural functions, strongly interrelated with each other. According to international and European definitions of rural areas there would be almost no rural area in Flanders. These international definitions are all developed to be used for analysis and policy within their specific context. They are not really applicable to Flanders because of the historical specificity of its spatial structure. Flanders is characterized by a giant urbanization pressure on its countryside while internationally rural depopulation is a point of interest. To date, for every single rural policy initiative – like the implementation of the European Rural Development Policy – Flanders used a specifically adapted definition, based on existing data or previously made delineations. To overcome this oversupply of definitions and delineations, the Flemish government funded a research project to obtain a clear and flexible definition of the Flemish countryside and a dynamic method to support Flemish rural policy aims. First, an analysis of the currently used definitions of the countryside in Flanders was made. It is clear that, depending on the perspective or the policy context, another definition of the countryside comes into view. The comparative study showed that, according to the used criteria, the area percentage of Flanders that is rural, varies between 9 and 93 per cent. Second, dynamic sets of criteria were developed, facilitating a flexible definition of the countryside, according to the policy aims concerned. This research part was focused on 6 policy themes, like ‘construction, maintenance and management of local (transport) infrastructures’ and ‘provision of (minimum) services (education, culture, health care, …)’. For each theme a dynamic set of criteria or indicators was constructed. These indicators make it possible to show where a policy theme manifests itself and/or where policy interventions are possible or needed. In this way every set of criteria makes up a new definition of rural Flanders. This method is dynamic; new data or insights can easily be incorporated and new criteria sets can be developed if other policy aims come into view. The developed method can contribute to a more region-oriented and theme-specific rural policy and funding mechanism

SurfelWarp: Efficient Non-Volumetric Single View Dynamic Reconstruction

We contribute a dense SLAM system that takes a live stream of depth images as input and reconstructs non-rigid deforming scenes in real time, without templates or prior models. In contrast to existing approaches, we do not maintain any volumetric data structures, such as truncated signed distance function (TSDF) fields or deformation fields, which are performance and memory intensive. Our system works with a flat point (surfel) based representation of geometry, which can be directly acquired from commodity depth sensors. Standard graphics pipelines and general purpose GPU (GPGPU) computing are leveraged for all central operations: i.e., nearest neighbor maintenance, non-rigid deformation field estimation and fusion of depth measurements. Our pipeline inherently avoids expensive volumetric operations such as marching cubes, volumetric fusion and dense deformation field update, leading to significantly improved performance. Furthermore, the explicit and flexible surfel based geometry representation enables efficient tackling of topology changes and tracking failures, which makes our reconstructions consistent with updated depth observations. Our system allows robots to maintain a scene description with non-rigidly deformed objects that potentially enables interactions with dynamic working environments.Comment: RSS 2018. The video and source code are available on https://sites.google.com/view/surfelwarp/hom