MTrainS: Improving DLRM training efficiency using heterogeneous memories

Recommendation models are very large, requiring terabytes (TB) of memory during training. In pursuit of better quality, the model size and complexity grow over time, which requires additional training data to avoid overfitting. This model growth demands a large number of resources in data centers. Hence, training efficiency is becoming considerably more important to keep the data center power demand manageable. In Deep Learning Recommendation Models (DLRM), sparse features capturing categorical inputs through embedding tables are the major contributors to model size and require high memory bandwidth. In this paper, we study the bandwidth requirement and locality of embedding tables in real-world deployed models. We observe that the bandwidth requirement is not uniform across different tables and that embedding tables show high temporal locality. We then design MTrainS, which leverages heterogeneous memory, including byte and block addressable Storage Class Memory for DLRM hierarchically. MTrainS allows for higher memory capacity per node and increases training efficiency by lowering the need to scale out to multiple hosts in memory capacity bound use cases. By optimizing the platform memory hierarchy, we reduce the number of nodes for training by 4-8X, saving power and cost of training while meeting our target training performance

Monitoring of Biodistribution and Persistence of Conditionally Replicative Adenovirus in a Murine Model of Ovarian Cancer Using Capsid-Incorporated mCherry and Expression of Human Somatostatin Receptor Subtype 2 Gene

A significant limiting factor to the human clinical application of conditionally replicative adenovirus (CRAd)-based virotherapy is the inability to noninvasively monitor these agents and their potential persistence. To address this issue, we proposed a novel imaging approach that combines transient expression of the human somatostatin receptor (SSTR) subtype 2 reporter gene with genetic labeling of the viral capsid with mCherry fluorescent protein. To test this dual modality system, we constructed the Ad5/3Δ24pIXcherry/SSTR CRAd and validated its capacity to generate fluorescent and nuclear signals in vitro and following intratumoral injection. Analysis of 64 Cu-CB-TE2A-Y3-TATE biodistribution in mice revealed reduced uptake in tumors injected with the imaging CRAd relative to the replication-incompetent, Ad-expressing SSTR2 but significantly greater uptake compared to the negative CRAd control. Optical imaging demonstrated relative correlation of fluorescent signal with virus replication as determined by viral genome quantification in tumors. Positron emission tomography/computed tomography studies demonstrated that we can visualize radioactive uptake in tumors injected with imaging CRAd and the trend for greater uptake by standardized uptake value analysis compared to control CRAd. In the aggregate, the plasticity of our dual imaging approach should provide the technical basis for monitoring CRAd biodistribution and persistence in preclinical studies while offering potential utility for a range of clinical applications

The Returns to Criminal Capital

Human capital theory (Becker 1962; Mincer 1958; Schultz 1960; 1961) posits that individuals can increase their labor market returns through investments in education and training. This concept has been studied extensively across several disciplines. An analog concept of criminal capital, while the focus of speculation and limited empirical study, remains considerably less developed theoretically and methodologically. This paper offers a formal theoretical model of criminal capital indicators and tests for greater illegal wage returns using a sample of serious adolescent offenders, many of whom participate in illegal income-generating activities. Our results reveal that, consistent with human capital theory, there are important illegal wage premiums associated with investments in criminal capital, notably an increasing but declining marginal return to experience and a premium for specialization. Further, as in studies of legal labor markets, we find strong evidence that, if left unaccounted for, non-random sample selection causes severe bias in models of illegal wages. Theoretical and practical implications of these results, along with directions for future research, are discussed

Social status strategy in early adolescent girls: Testosterone and value-based decision making

There has been strong interest, spanning several disciplines, in understanding adolescence as a developmental period of increased risk-taking behavior. Our goals focus on one line of investigation within this larger developmental risk framework. Specifically, we examined levels of pubertal hormones in girls in relation to their willingness to take greater financial risks to gain social status. To this end, we tested the hypothesis that higher levels of testosterone during the ages of pubertal maturation are associated with a greater willingness to sacrifice money for social admiration. Sixty-three girls ages 10-14 (Mage=12.74) participated in laboratory measures and completed at-home saliva sample collection. The Pubertal Development Scale (PDS) and basal hormone levels (testosterone, estradiol, DHEA) measured pubertal maturation. We made use of a developmentally appropriate version of an Auction Task in which adolescents could take financial risks in order to gain socially motivated outcomes (social status). PDS and testosterone were each associated with overall levels of financial risk taking over the course of the Auction Task. In hierarchical models, PDS and testosterone were predictors of the slope of overbidding over the course of the task. Results provide evidence for the role of testosterone and pubertal maturation in girls' motivations to engage in costly decision making in order to gain social status. Findings contribute to our understanding of the developmental underpinnings of some interesting aspects of adolescent risk behavior