LazyFox: Fast and parallelized overlapping community detection in large graphs

The detection of communities in graph datasets provides insight about a graph's underlying structure and is an important tool for various domains such as social sciences, marketing, traffic forecast, and drug discovery. While most existing algorithms provide fast approaches for community detection, their results usually contain strictly separated communities. However, most datasets would semantically allow for or even require overlapping communities that can only be determined at much higher computational cost. We build on an efficient algorithm, Fox, that detects such overlapping communities. Fox measures the closeness of a node to a community by approximating the count of triangles which that node forms with that community. We propose LazyFox, a multi-threaded version of the Fox algorithm, which provides even faster detection without an impact on community quality. This allows for the analyses of significantly larger and more complex datasets. LazyFox enables overlapping community detection on complex graph datasets with millions of nodes and billions of edges in days instead of weeks. As part of this work, LazyFox's implementation was published and is available as a tool under an MIT licence at https://github.com/TimGarrels/LazyFox.Comment: 17 pages, 5 figure

Multivariate sensor data analysis for oil refineries and multi-mode identification of system behavior in real-time

Large-scale oil refineries are equipped with mission-critical heavy machinery (boilers, engines, turbines, and so on) and are continuously monitored by thousands of sensors for process efficiency, environmental safety, and predictive maintenance purposes. However, sensors themselves are also prone to errors and failure. The quality of data received from these sensors should be verified before being used in system modeling. There is a need for reliable methods and systems that can provide data validation and reconciliation in real-time with high accuracy. In this paper, we develop a novel method for real-time data validation, gross error detection and classification over multivariate sensor data streams. The validated and high-quality data obtained from these processes is used for pattern analysis and modeling of industrial plants. We obtain sensor data from the power and petrochemical plants of an oil refinery and analyze them using various time-series modeling and data mining techniques that we integrate into a complex event processing engine. Next, we study the computational performance implications of the proposed methods and uncover regimes where they are sustainable over fast streams of sensor data. Finally, we detect shifts among steady-states of data, which represent systems' multiple operating modes and identify the time when a model reconstruction is required using DBSCAN clustering algorithm.Turkish Petroleum Refineries Inc. (TUPRAS) RD CenterPublisher versio

Real-time data reconciliation solutions for big data problems observed in oil refineries

Due to copyright restrictions, the access to the full text of this article is only available via subscription.Rafineriler tonlarca ham petrolün, her gün faklı kimyasal işlemden geçirilerek benzine ve diğer yan ürünlere dönüştürüldüğü dev endüstriyel tesislerdir. Bu makalede sensör-tabanlı petrol rafinelerine özel endüstriyel büyük veri problemleri tanımlanmakta (hacim hız çeşitlilik tutarsızlık) ve bu ortamlara uygun gerçek-zamanlı veri doğrulama ve uzlaştırma çözümleri sunulmaktadır."TÜPRAŞ ; TÜBİTA

Cyber-Security Gaps in a Digital Substation: From Sensors to SCADA

Development of digital substations provides power industrial operation, real-time functionalities and information access. A main challenge in DS is to ensure security, availability, and reliability of power systems as in conventional systems in addition to interoperability capability for different vendors. DS development is rather new in Norway and in an R&D Digital Substation pilot project ECODIS 1 1 Engineering and Condition monitoring in Digital Substation (ECODIS) is a project funded by Research Council of Norway (NFR), Innovation Project for the Industrial Sector - ENERGIX program, project number 296550, coordinated by Statnett R&D group., Statnett 2 2 Statnett is the system operator of the Norwegian power system. is investigating new functionality advantages and associated costs with IEC 61850 process bus technology. This paper examines cyber-security gaps and vulnerabilities introduced through digitalization of substations