6 research outputs found
Hyper: Distributed Cloud Processing for Large-Scale Deep Learning Tasks
Training and deploying deep learning models in real-world applications
require processing large amounts of data. This is a challenging task when the
amount of data grows to a hundred terabytes, or even, petabyte-scale. We
introduce a hybrid distributed cloud framework with a unified view to multiple
clouds and an on-premise infrastructure for processing tasks using both CPU and
GPU compute instances at scale. The system implements a distributed file system
and failure-tolerant task processing scheduler, independent of the language and
Deep Learning framework used. It allows to utilize unstable cheap resources on
the cloud to significantly reduce costs. We demonstrate the scalability of the
framework on running pre-processing, distributed training, hyperparameter
search and large-scale inference tasks utilizing 10,000 CPU cores and 300 GPU
instances with the overall processing power of 30 petaflops
PZnet: Efficient 3D ConvNet Inference on Manycore CPUs
Convolutional nets have been shown to achieve state-of-the-art accuracy in
many biomedical image analysis tasks. Many tasks within biomedical analysis
domain involve analyzing volumetric (3D) data acquired by CT, MRI and
Microscopy acquisition methods. To deploy convolutional nets in practical
working systems, it is important to solve the efficient inference problem.
Namely, one should be able to apply an already-trained convolutional network to
many large images using limited computational resources. In this paper we
present PZnet, a CPU-only engine that can be used to perform inference for a
variety of 3D convolutional net architectures. PZNet outperforms MKL-based CPU
implementations of PyTorch and Tensorflow by more than 3.5x for the popular
U-net architecture. Moreover, for 3D convolutions with low featuremap numbers,
cloud CPU inference with PZnet outperfroms cloud GPU inference in terms of cost
efficiency
Deep Lake: a Lakehouse for Deep Learning
Traditional data lakes provide critical data infrastructure for analytical
workloads by enabling time travel, running SQL queries, ingesting data with
ACID transactions, and visualizing petabyte-scale datasets on cloud storage.
They allow organizations to break down data silos, unlock data-driven
decision-making, improve operational efficiency, and reduce costs. However, as
deep learning takes over common analytical workflows, traditional data lakes
become less useful for applications such as natural language processing (NLP),
audio processing, computer vision, and applications involving non-tabular
datasets. This paper presents Deep Lake, an open-source lakehouse for deep
learning applications developed at Activeloop. Deep Lake maintains the benefits
of a vanilla data lake with one key difference: it stores complex data, such as
images, videos, annotations, as well as tabular data, in the form of tensors
and rapidly streams the data over the network to (a) Tensor Query Language, (b)
in-browser visualization engine, or (c) deep learning frameworks without
sacrificing GPU utilization. Datasets stored in Deep Lake can be accessed from
PyTorch, TensorFlow, JAX, and integrate with numerous MLOps tools