XEngine : Optimal Tensor Rematerialization for Neural Networks in Heterogeneous Environments

Memory efficiency is crucial in training deep learning networks on resource-restricted devices. During backpropagation, forward tensors are used to calculate gradients. Despite the option of keeping those dependencies in memory until they are reused in backpropagation, some forward tensors can be discarded and recomputed later from saved tensors, so-called checkpoints. This allows, in particular, for resource-constrained heterogeneous environments to make use of all available compute devices. Unfortunately, the definition of these checkpoints is a non-trivial problem and poses a challenge to the programmer—improper or excessive recomputations negate the benefit of checkpointing. In this article, we present XEngine, an approach that schedules network operators to heterogeneous devices in low memory environments by determining checkpoints and recomputations of tensors. Our approach selects suitable resources per timestep and operator and optimizes the end-to-end time for neural networks taking the memory limitation of each device into account. For this, we formulate a mixed-integer quadratic program (MIQP) to schedule operators of deep learning networks on heterogeneous systems. We compare our MIQP solver XEngine against Checkmate [12], a mixed-integer linear programming (MILP) approach that solves recomputation on a single device. Our solver finds solutions that are up to 22.5% faster than the fastest Checkmate schedule in which the network is computed exclusively on a single device. We also find valid schedules for networks making use of both central processing units and graphics processing units if memory limitations do not allow scheduling exclusively to the graphics processing unit

Race and Gender Differences of Body Physique and Self-Esteem in High School Students

This study investigated race and gender differences of body physique and self-esteem in high school students (N = 754, mean age = 16.5 + 1.2 yr) attending public schools. Participants completed a five part body physique questionnaire where responses were based on a set of nine figure silhouette drawings and a five part self esteem questionniare. African American females reported significantly larger body physique preferences when compared to their Caucasian counterparts (p0.05). African American’s (males and females) scored significantly higher on the self-esteem assessment than their Caucasian counterparts. These findings suggest that there might be important race and gender differences on various aspects of body physique and self-esteem in high school students

Barriers and Motivations to Exercise in Older African American and European American Women

Older adults, women, and minorities are the least active segments of the US population. The purpose of this study was to identify barriers and motivations to exercise associated with older African American and European American women. Eighty-nine European American and 115 African American women (mean age 72 years) completed a pencil-and-paper questionnaire composed of general demographic information and a list of 10 motivations and six barriers to exercise; participants were asked to check all barriers and motivations that applied to them. Significant ethnic differences were found for one barrier (too expensive) and one motivation (doctor’s orders). Significantly more European American women perceived expenses to be a barrier to exercise (χ 2 = 3.94,

A Quantitative Description for Optical Mass Measurement of Single Biomolecules

Label-free detection of single biomolecules in solution has been achieved using a variety of experimental approaches over the past decade. Yet, our understanding of the magnitude of the optical contrast and its relationship with the underlying atomic structure as well as the achievable measurement sensitivity and precision remain poorly defined. Here, we use a Fourier optics approach combined with an atomic structure-based molecular polarizability model to simulate mass photometry experiments from first principles. We find excellent agreement between several key experimentally determined parameters such as optical contrast-to-mass conversion, achievable mass accuracy, and molecular shape and orientation dependence. This allows us to determine detection sensitivity and measurement precision mostly independent of the optical detection approach chosen, resulting in a general framework for light-based single-molecule detection and quantification

Technical note: On uncertainties in plant water isotopic composition following extraction by cryogenic vacuum distillation

Recent studies have challenged the interpretation of plant water isotopes obtained through cryogenic vacuum distillation (CVD) based on observations of a large 2H fractionation. These studies have hypothesized the existence of an H-atom exchange between water and organic tissue during CVD extraction with the magnitude of H exchange related to relative water content of the sample; however, clear evidence is lacking. Here, we systematically tested the uncertainties in the isotopic composition of CVD-extracted water by conducting a series of incubation and rehydration experiments using isotopically depleted water, water at natural isotope abundance, woody materials with exchangeable H, and organic materials without exchangeable H (cellulose triacetate and caffeine). We show that the offsets between hydrogen and oxygen isotope ratios and expected reference values (Δ2H and Δ18O) have inversely proportional relationships with the absolute amount of water being extracted, i.e. the lower the water amount, the higher the Δ2H and Δ18O. However, neither Δ2H nor Δ18O values, were related to sample relative water content. The Δ2H pattern was more pronounced for materials with exchangeable H atoms than with non-exchangeable H atoms. This is caused by the combined effect of H exchange during the incubation of materials in water and isotopic enrichments during evaporation and sublimation that depend on absolute water amount. The H exchange during CVD extraction itself was negligible. Despite these technical issues, we observed that the water amount-dependent patterns were much less pronounced for samples at natural isotope abundance and particularly low when sufficiently high amounts of water were extracted (&gt;600 µL). Our study provides new insights into the mechanisms causing isotope fractionation during CVD extraction of water. The methodological uncertainties can be controlled if large samples of natural isotope abundance are used in ecohydrological studies.</p

Technical Note: On Uncertainties in Plant Water Isotopic Composition Following Extraction by Cryogenic Vacuum Distillation

Recent studies have challenged the interpretation of plant water isotopes obtained through cryogenic vacuum distillation (CVD) based on observations of a large 2H fractionation. These studies have hypothesized the existence of an H-atom exchange between water and organic tissue during CVD extraction with the magnitude of H exchange related to relative water content of the sample; however, clear evidence is lacking. Here, we systematically tested the uncertainties in the isotopic composition of CVD-extracted water by conducting a series of incubation and rehydration experiments using isotopically depleted water, water at natural isotope abundance, woody materials with exchangeable H, and organic materials without exchangeable H (cellulose triacetate and caffeine). We show that the offsets between hydrogen and oxygen isotope ratios and expected reference values (Δ2H and Δ18O) have inversely proportional relationships with the absolute amount of water being extracted, i.e. the lower the water amount, the higher the Δ2H and Δ18O. However, neither Δ2H nor Δ18O values, were related to sample relative water content. The Δ2H pattern was more pronounced for materials with exchangeable H atoms than with non-exchangeable H atoms. This is caused by the combined effect of H exchange during the incubation of materials in water and isotopic enrichments during evaporation and sublimation that depend on absolute water amount. The H exchange during CVD extraction itself was negligible. Despite these technical issues, we observed that the water amount-dependent patterns were much less pronounced for samples at natural isotope abundance and particularly low when sufficiently high amounts of water were extracted (\u3e600 µL). Our study provides new insights into the mechanisms causing isotope fractionation during CVD extraction of water. The methodological uncertainties can be controlled if large samples of natural isotope abundance are used in ecohydrological studies