4,274 research outputs found
Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI
Influenced by the great success of deep learning via cloud computing and the
rapid development of edge chips, research in artificial intelligence (AI) has
shifted to both of the computing paradigms, i.e., cloud computing and edge
computing. In recent years, we have witnessed significant progress in
developing more advanced AI models on cloud servers that surpass traditional
deep learning models owing to model innovations (e.g., Transformers, Pretrained
families), explosion of training data and soaring computing capabilities.
However, edge computing, especially edge and cloud collaborative computing, are
still in its infancy to announce their success due to the resource-constrained
IoT scenarios with very limited algorithms deployed. In this survey, we conduct
a systematic review for both cloud and edge AI. Specifically, we are the first
to set up the collaborative learning mechanism for cloud and edge modeling with
a thorough review of the architectures that enable such mechanism. We also
discuss potentials and practical experiences of some on-going advanced edge AI
topics including pretraining models, graph neural networks and reinforcement
learning. Finally, we discuss the promising directions and challenges in this
field.Comment: 20 pages, Transactions on Knowledge and Data Engineerin
APQ: Joint Search for Network Architecture, Pruning and Quantization Policy
We present APQ for efficient deep learning inference on resource-constrained
hardware. Unlike previous methods that separately search the neural
architecture, pruning policy, and quantization policy, we optimize them in a
joint manner. To deal with the larger design space it brings, a promising
approach is to train a quantization-aware accuracy predictor to quickly get the
accuracy of the quantized model and feed it to the search engine to select the
best fit. However, training this quantization-aware accuracy predictor requires
collecting a large number of quantized pairs, which involves
quantization-aware finetuning and thus is highly time-consuming. To tackle this
challenge, we propose to transfer the knowledge from a full-precision (i.e.,
fp32) accuracy predictor to the quantization-aware (i.e., int8) accuracy
predictor, which greatly improves the sample efficiency. Besides, collecting
the dataset for the fp32 accuracy predictor only requires to evaluate neural
networks without any training cost by sampling from a pretrained once-for-all
network, which is highly efficient. Extensive experiments on ImageNet
demonstrate the benefits of our joint optimization approach. With the same
accuracy, APQ reduces the latency/energy by 2x/1.3x over MobileNetV2+HAQ.
Compared to the separate optimization approach (ProxylessNAS+AMC+HAQ), APQ
achieves 2.3% higher ImageNet accuracy while reducing orders of magnitude GPU
hours and CO2 emission, pushing the frontier for green AI that is
environmental-friendly. The code and video are publicly available.Comment: Accepted by CVPR 202
Approachable Error Bounded Lossy Compression
Compression is commonly used in HPC applications to move and store data. Traditional lossless compression, however, does not provide adequate compression of floating point data often found in scientific codes. Recently, researchers and scientists have turned to lossy compression techniques that approximate the original data rather than reproduce it in order to achieve desired levels of compression. Typical lossy compressors do not bound the errors introduced into the data, leading to the development of error bounded lossy compressors (EBLC). These tools provide the desired levels of compression as mathematical guarantees on the errors introduced. However, the current state of EBLC leaves much to be desired. The existing EBLC all have different interfaces requiring codes to be changed to adopt new techniques; EBLC have many more configuration options than their predecessors, making them more difficult to use; and EBLC typically bound quantities like point wise errors rather than higher level metrics such as spectra, p-values, or test statistics that scientists typically use. My dissertation aims to provide a uniform interface to compression and to develop tools to allow application scientists to understand and apply EBLC. This dissertation proposal presents three groups of work: LibPressio, a standard interface for compression and analysis; FRaZ/LibPressio-Opt frameworks for the automated configuration of compressors using LibPressio; and work on tools for analyzing errors in particular domains
- …