Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models

Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers. In this paper we discuss the SW/HW co-designed solution for high-performance distributed training of large-scale DLRMs. We introduce a high-performance scalable software stack based on PyTorch and pair it with the new evolution of Zion platform, namely ZionEX. We demonstrate the capability to train very large DLRMs with up to 12 Trillion parameters and show that we can attain 40X speedup in terms of time to solution over previous systems. We achieve this by (i) designing the ZionEX platform with dedicated scale-out network, provisioned with high bandwidth, optimal topology and efficient transport (ii) implementing an optimized PyTorch-based training stack supporting both model and data parallelism (iii) developing sharding algorithms capable of hierarchical partitioning of the embedding tables along row, column dimensions and load balancing them across multiple workers; (iv) adding high-performance core operators while retaining flexibility to support optimizers with fully deterministic updates (v) leveraging reduced precision communications, multi-level memory hierarchy (HBM+DDR+SSD) and pipelining. Furthermore, we develop and briefly comment on distributed data ingestion and other supporting services that are required for the robust and efficient end-to-end training in production environments

Soft-computing-based false alarm reduction for hierarchical data of intrusion detection system

A false alarm rate of online anomaly-based intrusion detection system is a crucial concern. It is challenging to implement in the real-world scenarios when these anomalies occur sporadically. The existing intrusion detection system has been developed to limit or decrease the false alarm rate. However, the state-of-the-art approaches are attack or algorithm specific, which is not generic. In this article, a soft-computing-based approach has been designed to reduce the false-positive rate for hierarchical data of anomaly-based intrusion detection system. The recurrent neural network model is applied to classify the data set of intrusion detection system and normal instances for various subclasses. The designed approach is more practical, reason being, it does not require any assumption or knowledge of the data set structure. Experimental evaluation is conducted on various attacks on KDDCup’99 and NSL-KDD data sets. The proposed method enhances the intrusion detection systems that can work with data with dependent and independent features. Furthermore, this approach is also beneficial for real-life scenarios with a low occurrence of attacks

Not Available

Not AvailableFish cell line has emerged as an important tool in fishery biotechnology. In recent years, various fish cell lines have been developed by different researchers across the country. National Repository on Fish cell lines, established with the aim to preserve fish cell lines for training and education to stakeholders, has started functioning at National Bureau of Fish Genetic Resources, Lucknow. This repository is supposed to characterize and preserve the fish cell lines developed across the country and serve as a national referral centre for Indian and exotic fish cell lines. Currently, the repository is maintaining 50 fish cell lines deposited by various research institutes in India, including the cell lines developed at cell culture facility of National Bureau of Fish Genetic Resources. The cell lines have been successfully cryopreserved after verifying its authenticity by sequence analysis of two mitochondrial genes, viz. 16S rRNA and cytochrome c oxidase sub-unit I. Chromosomal analysis, transfection efficiency and immunocytochemistry are also being used to characterize the cell lines. The facility is serviceable for the collection, deposition and distribution of fish cell lines. This paper discusses the status as well as the methodology adopted for fish cell lines development, characterization and storage at NRFC.Department of Biotechnology, Government of India, New Delhi