4 research outputs found
Do people with different sociodemographic backgrounds value their health differently? Evaluating the role of positional objectivity
ObjectiveThe fundamental disconnect between the actual and the perceived health of an individual raises considerable skepticism on the self-reported health data as it may be confounded by an individual’s socio-economic status. In this light, the present study aims to assess if people with different sociodemographic backgrounds value their health differently.MethodsThe health-state valuation using time-trade off was performed in a cross-sectional survey among a representative sample of 2,311 adults from India. Individuals were selected using a multistage stratified random sampling from five Indian states to elicit their present health-state, and to perform the health-state valuation exercise using computer assisted personal interviewing. A single block of standardized health-states was valued by multiple individuals, each belonging to different socio-demographic group. The difference in the valuation of health was assessed using bivariate analysis. The impact of different sociodemographic factors on the health-state valuation was evaluated using Tobit regression model.ResultsDifferences in the valuation of health were observed among different groups of age, religion, family type, state of residence, substance abuse, presence of ailments at the time of valuation, and number of dependent members in the household. Even after controlling for the severity of the administered health states, factors having a significant association with the valuation of health are age, religion, state of residence, substance abuse, family type, number of dependent members in the household, and presence of chronic or both acute and chronic ailments. Younger individuals place a higher value to their health as compared to their older counterparts. As compared to a healthy individual, a person with ailments rates the same health-state as worse.ConclusionInequalities in self-reported ill-health cannot be attributed to positional objectivity; age, religion, state of residence, substance abuse, family type, dependents, and ailments impact individual health valuation
RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructure
We present RecD (Recommendation Deduplication), a suite of end-to-end
infrastructure optimizations across the Deep Learning Recommendation Model
(DLRM) training pipeline. RecD addresses immense storage, preprocessing, and
training overheads caused by feature duplication inherent in industry-scale
DLRM training datasets. Feature duplication arises because DLRM datasets are
generated from interactions. While each user session can generate multiple
training samples, many features' values do not change across these samples. We
demonstrate how RecD exploits this property, end-to-end, across a deployed
training pipeline. RecD optimizes data generation pipelines to decrease dataset
storage and preprocessing resource demands and to maximize duplication within a
training batch. RecD introduces a new tensor format, InverseKeyedJaggedTensors
(IKJTs), to deduplicate feature values in each batch. We show how DLRM model
architectures can leverage IKJTs to drastically increase training throughput.
RecD improves the training and preprocessing throughput and storage efficiency
by up to 2.48x, 1.79x, and 3.71x, respectively, in an industry-scale DLRM
training system.Comment: Published in the Proceedings of the Sixth Conference on Machine
Learning and Systems (MLSys 2023
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models
Deep learning recommendation models (DLRMs) are used across many
business-critical services at Facebook and are the single largest AI
application in terms of infrastructure demand in its data-centers. In this
paper we discuss the SW/HW co-designed solution for high-performance
distributed training of large-scale DLRMs. We introduce a high-performance
scalable software stack based on PyTorch and pair it with the new evolution of
Zion platform, namely ZionEX. We demonstrate the capability to train very large
DLRMs with up to 12 Trillion parameters and show that we can attain 40X speedup
in terms of time to solution over previous systems. We achieve this by (i)
designing the ZionEX platform with dedicated scale-out network, provisioned
with high bandwidth, optimal topology and efficient transport (ii) implementing
an optimized PyTorch-based training stack supporting both model and data
parallelism (iii) developing sharding algorithms capable of hierarchical
partitioning of the embedding tables along row, column dimensions and load
balancing them across multiple workers; (iv) adding high-performance core
operators while retaining flexibility to support optimizers with fully
deterministic updates (v) leveraging reduced precision communications,
multi-level memory hierarchy (HBM+DDR+SSD) and pipelining. Furthermore, we
develop and briefly comment on distributed data ingestion and other supporting
services that are required for the robust and efficient end-to-end training in
production environments