445 research outputs found

    Lightweight Asynchronous Snapshots for Distributed Dataflows

    Full text link
    Distributed stateful stream processing enables the deployment and execution of large scale continuous computations in the cloud, targeting both low latency and high throughput. One of the most fundamental challenges of this paradigm is providing processing guarantees under potential failures. Existing approaches rely on periodic global state snapshots that can be used for failure recovery. Those approaches suffer from two main drawbacks. First, they often stall the overall computation which impacts ingestion. Second, they eagerly persist all records in transit along with the operation states which results in larger snapshots than required. In this work we propose Asynchronous Barrier Snapshotting (ABS), a lightweight algorithm suited for modern dataflow execution engines that minimises space requirements. ABS persists only operator states on acyclic execution topologies while keeping a minimal record log on cyclic dataflows. We implemented ABS on Apache Flink, a distributed analytics engine that supports stateful stream processing. Our evaluation shows that our algorithm does not have a heavy impact on the execution, maintaining linear scalability and performing well with frequent snapshots.Comment: 8 pages, 7 figure

    Geospatial Data Management Research: Progress and Future Directions

    Get PDF
    Without geospatial data management, today®s challenges in big data applications such as earth observation, geographic information system/building information modeling (GIS/BIM) integration, and 3D/4D city planning cannot be solved. Furthermore, geospatial data management plays a connecting role between data acquisition, data modelling, data visualization, and data analysis. It enables the continuous availability of geospatial data and the replicability of geospatial data analysis. In the first part of this article, five milestones of geospatial data management research are presented that were achieved during the last decade. The first one reflects advancements in BIM/GIS integration at data, process, and application levels. The second milestone presents theoretical progress by introducing topology as a key concept of geospatial data management. In the third milestone, 3D/4D geospatial data management is described as a key concept for city modelling, including subsurface models. Progress in modelling and visualization of massive geospatial features on web platforms is the fourth milestone which includes discrete global grid systems as an alternative geospatial reference framework. The intensive use of geosensor data sources is the fifth milestone which opens the way to parallel data storage platforms supporting data analysis on geosensors. In the second part of this article, five future directions of geospatial data management research are presented that have the potential to become key research fields of geospatial data management in the next decade. Geo-data science will have the task to extract knowledge from unstructured and structured geospatial data and to bridge the gap between modern information technology concepts and the geo-related sciences. Topology is presented as a powerful and general concept to analyze GIS and BIM data structures and spatial relations that will be of great importance in emerging applications such as smart cities and digital twins. Data-streaming libraries and “in-situ” geo-computing on objects executed directly on the sensors will revolutionize geo-information science and bridge geo-computing with geospatial data management. Advanced geospatial data visualization on web platforms will enable the representation of dynamically changing geospatial features or moving objects’ trajectories. Finally, geospatial data management will support big geospatial data analysis, and graph databases are expected to experience a revival on top of parallel and distributed data stores supporting big geospatial data analysis

    Critical Overview of Loops and Foams

    Full text link
    This is a review of the present status of loop and spin foam approaches to quantization of four-dimensional general relativity. It aims at raising various issues which seem to challenge some of the methods and the results often taken as granted in these domains. A particular emphasis is given to the issue of diffeomorphism and local Lorentz symmetries at the quantum level and to the discussion of new spin foam models. We also describe modifications of these two approaches which may overcome their problems and speculate on other promising research directions.Comment: 75 page

    Locally Collapsed 3-Manifolds

    Full text link
    We prove that a 3-dimensional compact Riemannian manifold which is locally collapsed, with respect to a lower curvature bound, is a graph manifold. This theorem was stated by Perelman and was used in his proof of the geometrization conjecture.Comment: Final versio

    Discrete-time gradient flows and law of large numbers in Alexandrov spaces

    Get PDF
    We develop the theory of discrete-time gradient flows for convex functions on Alexandrov spaces with arbitrary upper or lower curvature bounds. We employ different resolvent maps in the upper and lower curvature bound cases to construct such a flow, and show its convergence to a minimizer of the potential function. We also prove a stochastic version, a generalized law of large numbers for convex function valued random variables, which not only extends Sturm's law of large numbers on nonpositively curved spaces to arbitrary lower or upper curvature bounds, but this version seems new even in the Euclidean setting. These results generalize those in nonpositively curved spaces (partly for squared distance functions) due to Ba\v{c}\'ak, Jost, Sturm and others, and the lower curvature bound case seems entirely new.Comment: 28 pages; minor corrections; to appear in Calc. Var. PD
    • 

    corecore