2 research outputs found
Managing Machine Learning Workflow Components
Machine Learning Workflows (MLWfs) have become essential and a disruptive
approach in problem-solving over several industries. However, the development
process of MLWfs may be complicated, hard to achieve, time-consuming, and
error-prone. To handle this problem, in this paper, we introduce machine
learning workflow management (MLWfM) as a technique to aid the development and
reuse of MLWfs and their components through three aspects: representation,
execution, and creation. More precisely, we discuss our approach to structure
the MLWfs' components and their metadata to aid retrieval and reuse of
components in new MLWfs. Also, we consider the execution of these components
within a tool. The hybrid knowledge representation, called Hyperknowledge,
frames our methodology, supporting the three MLWfM's aspects. To validate our
approach, we show a practical use case in the Oil & Gas industry.Comment: 12 pages, 3 figures, to appear at the International Journal of
Semantic Computing, 14(2), 202
Workflow Provenance in the Lifecycle of Scientific Machine Learning
Machine Learning (ML) has already fundamentally changed several businesses.
More recently, it has also been profoundly impacting the computational science
and engineering domains, like geoscience, climate science, and health science.
In these domains, users need to perform comprehensive data analyses combining
scientific data and ML models to provide for critical requirements, such as
reproducibility, model explainability, and experiment data understanding.
However, scientific ML is multidisciplinary, heterogeneous, and affected by the
physical constraints of the domain, making such analyses even more challenging.
In this work, we leverage workflow provenance techniques to build a holistic
view to support the lifecycle of scientific ML. We contribute with (i)
characterization of the lifecycle and taxonomy for data analyses; (ii) design
principles to build this view, with a W3C PROV compliant data representation
and a reference system architecture; and (iii) lessons learned after an
evaluation in an Oil & Gas case using an HPC cluster with 393 nodes and 946
GPUs. The experiments show that the principles enable queries that integrate
domain semantics with ML models while keeping low overhead (<1%), high
scalability, and an order of magnitude of query acceleration under certain
workloads against without our representation.Comment: 21 pages, 10 figures, Under review in a scientific journal since June
30th, 2020. arXiv admin note: text overlap with arXiv:1910.0422