Do people with different sociodemographic backgrounds value their health differently? Evaluating the role of positional objectivity

ObjectiveThe fundamental disconnect between the actual and the perceived health of an individual raises considerable skepticism on the self-reported health data as it may be confounded by an individual’s socio-economic status. In this light, the present study aims to assess if people with different sociodemographic backgrounds value their health differently.MethodsThe health-state valuation using time-trade off was performed in a cross-sectional survey among a representative sample of 2,311 adults from India. Individuals were selected using a multistage stratified random sampling from five Indian states to elicit their present health-state, and to perform the health-state valuation exercise using computer assisted personal interviewing. A single block of standardized health-states was valued by multiple individuals, each belonging to different socio-demographic group. The difference in the valuation of health was assessed using bivariate analysis. The impact of different sociodemographic factors on the health-state valuation was evaluated using Tobit regression model.ResultsDifferences in the valuation of health were observed among different groups of age, religion, family type, state of residence, substance abuse, presence of ailments at the time of valuation, and number of dependent members in the household. Even after controlling for the severity of the administered health states, factors having a significant association with the valuation of health are age, religion, state of residence, substance abuse, family type, number of dependent members in the household, and presence of chronic or both acute and chronic ailments. Younger individuals place a higher value to their health as compared to their older counterparts. As compared to a healthy individual, a person with ailments rates the same health-state as worse.ConclusionInequalities in self-reported ill-health cannot be attributed to positional objectivity; age, religion, state of residence, substance abuse, family type, dependents, and ailments impact individual health valuation

RecD: Deduplication for End-to-End Deep Learning Recommendation Model Training Infrastructure

We present RecD (Recommendation Deduplication), a suite of end-to-end infrastructure optimizations across the Deep Learning Recommendation Model (DLRM) training pipeline. RecD addresses immense storage, preprocessing, and training overheads caused by feature duplication inherent in industry-scale DLRM training datasets. Feature duplication arises because DLRM datasets are generated from interactions. While each user session can generate multiple training samples, many features' values do not change across these samples. We demonstrate how RecD exploits this property, end-to-end, across a deployed training pipeline. RecD optimizes data generation pipelines to decrease dataset storage and preprocessing resource demands and to maximize duplication within a training batch. RecD introduces a new tensor format, InverseKeyedJaggedTensors (IKJTs), to deduplicate feature values in each batch. We show how DLRM model architectures can leverage IKJTs to drastically increase training throughput. RecD improves the training and preprocessing throughput and storage efficiency by up to 2.48x, 1.79x, and 3.71x, respectively, in an industry-scale DLRM training system.Comment: Published in the Proceedings of the Sixth Conference on Machine Learning and Systems (MLSys 2023

Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models

Deep learning recommendation models (DLRMs) are used across many business-critical services at Facebook and are the single largest AI application in terms of infrastructure demand in its data-centers. In this paper we discuss the SW/HW co-designed solution for high-performance distributed training of large-scale DLRMs. We introduce a high-performance scalable software stack based on PyTorch and pair it with the new evolution of Zion platform, namely ZionEX. We demonstrate the capability to train very large DLRMs with up to 12 Trillion parameters and show that we can attain 40X speedup in terms of time to solution over previous systems. We achieve this by (i) designing the ZionEX platform with dedicated scale-out network, provisioned with high bandwidth, optimal topology and efficient transport (ii) implementing an optimized PyTorch-based training stack supporting both model and data parallelism (iii) developing sharding algorithms capable of hierarchical partitioning of the embedding tables along row, column dimensions and load balancing them across multiple workers; (iv) adding high-performance core operators while retaining flexibility to support optimizers with fully deterministic updates (v) leveraging reduced precision communications, multi-level memory hierarchy (HBM+DDR+SSD) and pipelining. Furthermore, we develop and briefly comment on distributed data ingestion and other supporting services that are required for the robust and efficient end-to-end training in production environments