49 research outputs found
GraphScope Flex: LEGO-like Graph Computing Stack
Graph computing has become increasingly crucial in processing large-scale
graph data, with numerous systems developed for this purpose. Two years ago, we
introduced GraphScope as a system addressing a wide array of graph computing
needs, including graph traversal, analytics, and learning in one system. Since
its inception, GraphScope has achieved significant technological advancements
and gained widespread adoption across various industries. However, one key
lesson from this journey has been understanding the limitations of a
"one-size-fits-all" approach, especially when dealing with the diversity of
programming interfaces, applications, and data storage formats in graph
computing. In response to these challenges, we present GraphScope Flex, the
next iteration of GraphScope. GraphScope Flex is designed to be both
resource-efficient and cost-effective, while also providing flexibility and
user-friendliness through its LEGO-like modularity. This paper explores the
architectural innovations and fundamental design principles of GraphScope Flex,
all of which are direct outcomes of the lessons learned during our ongoing
development process. We validate the adaptability and efficiency of GraphScope
Flex with extensive evaluations on synthetic and real-world datasets. The
results show that GraphScope Flex achieves 2.4X throughput and up to 55.7X
speedup over other systems on the LDBC Social Network and Graphalytics
benchmarks, respectively. Furthermore, GraphScope Flex accomplishes up to a
2,400X performance gain in real-world applications, demonstrating its
proficiency across a wide range of graph computing scenarios with increased
effectiveness
Survey of scientific programming techniques for the management of data-intensive engineering environments
The present paper introduces and reviews existing technology and research works in the field of scientific programming methods and techniques in data-intensive engineering environments. More specifically, this survey aims to collect those relevant approaches that have faced the challenge of delivering more advanced and intelligent methods taking advantage of the existing large datasets. Although existing tools and techniques have demonstrated their ability to manage complex engineering processes for the development and operation of safety-critical systems, there is an emerging need to know how existing computational science methods will behave to manage large amounts of data. That is why, authors review both existing open issues in the context of engineering with special focus on scientific programming techniques and hybrid approaches. 1193 journal papers have been found as the representative in these areas screening 935 to finally make a full review of 122. Afterwards, a comprehensive mapping between techniques and engineering and nonengineering domains has been conducted to classify and perform a meta-analysis of the current state of the art. As the main result of this work, a set of 10 challenges for future data-intensive engineering environments have been outlined.The current work has been partially supported by the Research Agreement between the RTVE (the Spanish Radio and Television Corporation) and the UC3M to boost research in the field of Big Data, Linked Data, Complex Network Analysis, and Natural Language. It has also received the support of the Tecnologico Nacional de Mexico (TECNM), National Council of Science and Technology (CONACYT), and the Public Education Secretary (SEP) through PRODEP
Recommended from our members
Network Structures, Concurrency, and Interpretability: Lessons from the Development of an AI Enabled Graph Database System
This thesis describes the development of the SmartGraph, an AI enabled graph database. The need for such a system has been independently recognized in the isolated fields of graph databases, graph computing, and computational graph deep learning systems, such as TensorFlow. Though prior works have investigated some relationships between these fields, we believe that the SmartGraph is the first system designed from conception to incorporate the most significant and useful characteristics of each. Examples include the ability to store graph structured data, run analytics natively on this data, and run gradient descent algorithms. It is the synergistic aspects of combining these fields that provide the most novel results presented in this dissertation. Key among them is how the notion of âgraph queryingâ as used in graph databases can be used to solve a problem that has plagued deep learning systems since their inception; rather than attempting to embed graph structured datasets into restrictive vector spaces, we instead allow the deep learning functionality of the system to natively perform graph querying in memory during optimization as a way of interpreting (and learning) the graph. This results in a concept of natural and interpretable processing of graph structured data.
Graph computing systems have traditionally used distributed computing across multiple compute nodes (e.g. separate machines connected via Ethernet or internet) to deal with large-scale datasets whilst working sequentially on problems over entire datasets. In this dissertation, we outline a distributed graph computing methodology that facilitates all the above capabilities (even in an environment consisting of a single physical machine) while allowing for a workflow more typical of a graph database than a graph computing system; massive concurrent access allowing for arbitrarily asynchronous execution of queries and analytics across the entire system. Further, we demonstrate how this methodology is key to the artificial intelligence capabilities of the system
The blessings of explainable AI in operations & maintenance of wind turbines
Wind turbines play an integral role in generating clean energy, but regularly suffer from operational inconsistencies and failures leading to unexpected downtimes and significant Operations & Maintenance (O&M) costs. Condition-Based Monitoring (CBM) has been utilised in the past to monitor operational inconsistencies in turbines by applying signal processing techniques to vibration data. The last decade has witnessed growing interest in leveraging Supervisory Control & Acquisition (SCADA) data from turbine sensors towards CBM. Machine Learning (ML) techniques have been utilised to predict incipient faults in turbines and forecast vital operational parameters with high accuracy by leveraging SCADA data and alarm logs. More recently, Deep Learning (DL) methods have outperformed conventional ML techniques, particularly for anomaly prediction. Despite demonstrating immense promise in transitioning to Artificial Intelligence (AI), such models are generally black-boxes that cannot provide rationales behind their predictions, hampering the ability of turbine operators to rely on automated decision making. We aim to help combat this challenge by providing a novel perspective on Explainable AI (XAI) for trustworthy decision support.This thesis revolves around three key strands of XAI â DL, Natural Language Generation (NLG) and Knowledge Graphs (KGs), which are investigated by utilising data from an operational turbine. We leverage DL and NLG to predict incipient faults and alarm events in the turbine in natural language as well as generate human-intelligible O&M strategies to assist engineers in fixing/averting the faults. We also propose specialised DL models which can predict causal relationships in SCADA features as well as quantify the importance of vital parameters leading to failures. The thesis finally culminates with an interactive Question- Answering (QA) system for automated reasoning that leverages multimodal domain-specific information from a KG, facilitating engineers to retrieve O&M strategies with natural language questions. By helping make turbines more reliable, we envisage wider adoption of wind energy sources towards tackling climate change
Modern temporal network theory: A colloquium
The power of any kind of network approach lies in the ability to simplify a
complex system so that one can better understand its function as a whole.
Sometimes it is beneficial, however, to include more information than in a
simple graph of only nodes and links. Adding information about times of
interactions can make predictions and mechanistic understanding more accurate.
The drawback, however, is that there are not so many methods available, partly
because temporal networks is a relatively young field, partly because it more
difficult to develop such methods compared to for static networks. In this
colloquium, we review the methods to analyze and model temporal networks and
processes taking place on them, focusing mainly on the last three years. This
includes the spreading of infectious disease, opinions, rumors, in social
networks; information packets in computer networks; various types of signaling
in biology, and more. We also discuss future directions.Comment: Final accepted versio
Learning Logical Rules from Knowledge Graphs
Ph.D. (Integrated) ThesisExpressing and extracting regularities in multi-relational data, where data points are interrelated
and heterogeneous, requires well-designed knowledge representation. Knowledge Graphs (KGs),
as a graph-based representation of multi-relational data, have seen a rapidly growing presence in
industry and academia, where many real-world applications and academic research are either
enabled or augmented through the incorporation of KGs. However, due to the way KGs are
constructed, they are inherently noisy and incomplete. In this thesis, we focus on developing
logic-based graph reasoning systems that utilize logical rules to infer missing facts for the
completion of KGs. Unlike most rule learners that primarily mine abstract rules that contain
no constants, we are particularly interested in learning instantiated rules that contain constants
due to their ability to represent meaningful patterns and correlations that can not be expressed
by abstract rules. The inclusion of instantiated rules often leads to exponential growth in the
search space. Therefore, it is necessary to develop optimization strategies to balance between
scalability and expressivity. To such an end, we propose GPFL, a probabilistic rule learning
system optimized to mine instantiated rules through the implementation of a novel two-stage
rule generation mechanism. Through experiments, we demonstrate that GPFL not only performs
competitively on knowledge graph completion but is also much more efficient then existing
methods at mining instantiated rules. With GPFL, we also reveal overfitting instantiated rules
and provide detailed analyses about their impact on system performance. Then, we propose RHF,
a generic framework for constructing rule hierarchies from a given set of rules. We demonstrate
through experiments that with RHF and the hierarchical pruning techniques enabled by it,
significant reductions in runtime and rule size are observed due to the pruning of unpromising
rules. Eventually, to test the practicability of rule learning systems, we develop Ranta, a novel
drug repurposing system that relies on logical rules as features to make interpretable inferences.
Ranta outperforms existing methods by a large margin in predictive performance and can make
reasonable repurposing suggestions with interpretable evidence
Sense and Respond
Over the past century, the manufacturing industry has undergone a number of paradigm shifts: from the Ford assembly line (1900s) and its focus on efficiency to the Toyota production system (1960s) and its focus on effectiveness and JIDOKA; from flexible manufacturing (1980s) to reconfigurable manufacturing (1990s) (both following the trend of mass customization); and from agent-based manufacturing (2000s) to cloud manufacturing (2010s) (both deploying the value stream complexity into the material and information flow, respectively). The next natural evolutionary step is to provide value by creating industrial cyber-physical assets with human-like intelligence. This will only be possible by further integrating strategic smart sensor technology into the manufacturing cyber-physical value creating processes in which industrial equipment is monitored and controlled for analyzing compression, temperature, moisture, vibrations, and performance. For instance, in the new wave of the âIndustrial Internet of Thingsâ (IIoT), smart sensors will enable the development of new applications by interconnecting software, machines, and humans throughout the manufacturing process, thus enabling suppliers and manufacturers to rapidly respond to changing standards. This reprint of âSense and Respondâ aims to cover recent developments in the field of industrial applications, especially smart sensor technologies that increase the productivity, quality, reliability, and safety of industrial cyber-physical value-creating processes