Reward Scale Robustness for Proximal Policy Optimization via DreamerV3 Tricks

Most reinforcement learning methods rely heavily on dense, well-normalized environment rewards. DreamerV3 recently introduced a model-based method with a number of tricks that mitigate these limitations, achieving state-of-the-art on a wide range of benchmarks with a single set of hyperparameters. This result sparked discussion about the generality of the tricks, since they appear to be applicable to other reinforcement learning algorithms. Our work applies DreamerV3's tricks to PPO and is the first such empirical study outside of the original work. Surprisingly, we find that the tricks presented do not transfer as general improvements to PPO. We use a high quality PPO reference implementation and present extensive ablation studies totaling over 10,000 A100 hours on the Arcade Learning Environment and the DeepMind Control Suite. Though our experiments demonstrate that these tricks do not generally outperform PPO, we identify cases where they succeed and offer insight into the relationship between the implementation tricks. In particular, PPO with these tricks performs comparably to PPO on Atari games with reward clipping and significantly outperforms PPO without reward clipping.Comment: Accepted to NeurIPS 202

Evaluation of a cognitive affective model of physical activity behavior

© 2020 The Author(s). Background: To empirically evaluate a cognitive affective model of physical activity. This bidirectional, cyclical model hypotheses that executive control processes directly influence habitual engagement in exercise and also directly subserve the exercise-induced affective response to acute exercise associated with future physical activity. Methods: The present study employed a one-week prospective, multi-site design. Participant recruitment and data collection occurred at two separate University sites (one in the United States and the other in Canada). Participants completed a bout of treadmill exercise, with affect and arousal assessed before, during and after the bout of exercise. Subjective and objective measures of executive function were assessed during this visit. Following this laboratory visit, seven days of accelerometry were employed to measure habitual engagement in physical activity. Results: Within our inactive, young adult sample, we observed some evidence of 1) aspects of executive function were associated with more light-intensity physical activity in the future (1-week later) (r = 0.36, 95% CI = -0.03 to 0.66, P = 0.07), 2) aspects of executive function were associated with post-exercise affect (r = -0.39, 95% CI = -0.67 to -0.03, P = 0.03) and forecasted affect (r = 0.47, 95% CI = 0.11 to 0.72, P = 0.01), and 3) aspects of acute exercise arousal and affect were associated with current mild-intensity physical activity behavior (r = 0.41, 95% CI = 0.04 to 0.68, P = 0.03). Conclusion: We demonstrate partial support of a cognitive-affective model of physical activity

Relationship Between Body Composition, Body Fat Distribution, and Blood Lipids Among Law Enforcement Officers: Part 2

Law enforcement officers (LEOs) have a high-stress occupation, which is prone to cardiovascular disease (CVD). Data suggest a 1.7-fold higher CVD prevalence versus the general public, in addition to 40.5% of LEOs being classified as obese. Emerging evidence suggests that lipid-related atherosclerotic risk may be better captured by including advanced blood lipid panels (i.e., cholesterol particle type and size). However, there needs to be more research regarding the relationship between body composition, body fat distribution, and advanced blood lipid panels concerning CVD risk in LEOs. PURPOSE: To determine if body composition and fat distribution measures correlate with predictive advanced lipid markers in LEOs. METHODS: Forty-three LEOs (male: n=40; Female: n=3; age = 41.7±9.6 yrs; weight = 91.9±15.4 kg; height = 179.8±8.7 cm; VO2max: 37.0±6.1 ml/kg/min) from a local police department were evaluated. Fasting blood samples were collected to assess biomarkers of CVD risk: number of low-density lipoprotein particles (LDL-p), small LDL-p, LDL size (LDL-s), number of high-density lipoprotein particles (HDL-p), large HDL-p, HDL size (HDL-s), number of very low-density lipoprotein particles (VLDL-p), and VLDL size (VLDL-s). Dual-energy x-ray absorptiometry was used to measure body composition. Bivariate Pearson correlation matrix and ordinary least squares (OLS) regression analyses were used to examine the relationship between body composition and fat distribution. RESULTS: The Bivariate Pearson correlation matrix revealed higher body weight, lean mass, visceral adipose tissue, and gynoid adiposity correlated with several advanced lipid biomarkers. The OLS regression analysis revealed Body weight to be positively predictive (p\u3c0.05) of LDL-p and small LDL-p but inversely predictive (p\u3c0.05) of LDL-s, large HDL-p, and HDL-s; Lean mass to be positively predictive of LDL-p, small LDL-p, but inversely predictive of LDL-s, HDL-p, large HDL-p, and HDL-s; Visceral adipose tissue to be positively predictive of small LDL-p and large VLDL-p, but inversely predictive of LDL-s, large HDL-p, and HDL-s; and gynoid fat distribution to be positively predictive of HDL-p, large HDL-p, and HDL-s. CONCLUSION: Changes in body composition seen in LEOs with increased weight and fat distribution showed correlations with advanced blood lipid markers, which can be predictive of high CVD risk and other potential medical conditions. These data provide insight into the association of body composition and fat distribution with markers of CVD risk (i.e., advanced blood lipids)

Relationship Between Body Composition, Body Fat Distribution, and Blood Lipids Among Law Enforcement Officers: Part 1

Law enforcement officers (LEOs) have a high-stress occupation which is prone to cardiovascular disease (CVD). In fact, data suggest that LEOs have a 1.7-fold higher CVD prevalence versus the general public, in addition to 40.5% of LEOs being classified as obese. However, research is lacking regarding the relationship between body composition, body fat distribution, and blood lipid panels as it pertains to CVD risk in LEOs. PURPOSE: To determine if body composition and fat distribution measures correlate with predictive lipid markers in LEOs. METHODS: Forty-three LEOs (age = 41.7±9.6 yrs; weight = 91.9±15.4 kg; height = 179.8±8.7 cm; VO2max: 37.0±6.16 ml/kg/min) from a local police department were evaluated. Fasting blood samples were collected to assess biomarkers of CVD risk: low-density lipoprotein (LDL), high-density lipoprotein (HDL), total cholesterol (TC), and triglycerides (TG). Dual-energy x-ray absorptiometry was used to measure body composition and body fat distribution. Bivariate Pearson correlation matrix was used to determine correlations (p\u3c0.05* and p\u3c0.01**). To further assess the relationship between body composition, fat distribution measures, and blood lipids, ordinary least square (OLS) regression analyses were used. RESULTS: Lower body weight correlated with greater HDL concentrations (r=-0.432**). Higher fat mass correlated with greater TG concentrations (r=0.338*), while greater lean mass was inversely correlated with HDL concentrations (r=-0.496**). Android and gynoid adiposity were positively correlated with greater TG (r=0.359*) and HDL (r=0.320*) concentrations, respectively. Lastly, higher visceral adipose tissue was correlated with greater TG concentrations (r=0.430**). The OLS regression analysis revealed (p\u3c0.05) 1) weight was inversely predictive of HDL, 2) Fat mass was positively predictive of TG, 3) lean mass was inversely predictive of HDL, 4) android adiposity was positively predictive of TG, 5) gynoid adiposity was positively predictive of HDL, and 6) visceral adipose tissue was positively predictive of TG. CONCLUSION: Measures of body composition seen in LEOs with increased body fat showed positive correlations with blood lipid markers (TG and HDL), which can be predictive of high CVD risk and other potential medical conditions. These data provide insight into the association of body composition and fat distribution with markers of CVD risk

Risk and protective factors of social networks on alcohol, cannabis, and opioid use among urban American Indian/Alaska Native emerging adults.

OBJECTIVES: Assess associations between social networks and urban American Indian/Alaska Native emerging adults alcohol, cannabis, and opioid use and intentions. METHOD: American Indian/Alaska Native participants ages 18-25 (N = 150; 86% female) were recruited across the United States from 12/20 to 10/21 via social media. Participants named up to 15 people whom they talked with most over the past 3 months and reported who (a) used alcohol and cannabis heavily or used other drugs (e.g., opioid use), (b) engaged in traditional practices, and (c) provided support. They also reported past 3-month alcohol, cannabis, and opioid use and intentions to use. RESULTS: Having a higher proportion of network members engaging in regular cannabis and heavy alcohol use (but not other drugs) was associated with more frequent cannabis use and stronger cannabis use intentions. Participants with higher proportions of members engaging in heavy alcohol use, regular cannabis use, or other drug use and who did not engage in traditional practices were more likely to report cannabis use and greater intentions to use cannabis and drink alcohol. In contrast, participants with higher proportions of network members engaging in traditional practices and who did not report heavy alcohol use, regular cannabis use, or other drug use were less likely to report intentions to use cannabis or drink alcohol. CONCLUSIONS: Findings emphasize what many studies have shown among various racial/ethnic groups-having network members who use substances increases the chance of use. Findings also highlight that traditional practices may be an important part of the prevention approach for this population. (PsycInfo Database Record (c) 2023 APA, all rights reserved)

Recovery using “float” from high intensity stress on growth hormone-like molecules in resistance trained men

Objective The purpose of this study was to examine the influence of a novel “floatation-restricted environmental stimulation therapy” (floatation-REST) on growth hormone responses to an intense resistance exercise stress. Design Nine resistance trained men (age: 23.4 ± 2.5 yrs.; height: 175.3 ± 5.4 cm; body mass: 85.3 ± 7.9 kg) completed a balanced, crossover-controlled study design with two identical exercise trials, differing only in post-exercise recovery intervention (i.e., control or floatation-REST). A two-week washout period was used between experimental conditions. Plasma lactate was measured pre-exercise, immediately post-exercise and after the 1 h. recovery interventions. Plasma iGH was measured pre-exercise, immediately-post exercise, and after the recovery intervention, as well as 24 h and 48 h after the exercise test. The bGH-L was measured only at pre-exercise and following each recovery intervention. Results For both experimental conditions, a significant (P ≤ 0.05) increase in lactate concentrations were observed immediately post-exercise (~14 mmol • L-1) and remained slightly elevated after the recovery condition. The same pattern of responses was observed for iGH with no differences from resting values at 24 and 48 h of recovery. The bGH-L showed no exercise-induced changes following recovery with either treatment condition, however concentration values were dramatically lower than ever reported. Conclusion The use of floatation-REST therapy immediately following intense resistance exercise does not appear to influence anterior pituitary function in highly resistance trained men. However, the lower values of bGH suggest dramatically different molecular processing mechanisms at work in this highly trained population

Effects of Human Milk Oligosaccharide 2-Fucosyllactose Ingestion on Weight Loss and Markers of Health.

BACKGROUND: 2-Fucosyllactose (2-FL) is an oligosaccharide contained in human milk and possesses prebiotic and anti-inflammatory effects, which may alleviate skeletal muscle atrophy under caloric restriction. This study evaluated the impacts of 12 weeks of 2-FL supplementation in conjunction with exercise (10,000 steps/day, 5 days/week) and energy-reduced (-300 kcals/day) dietary interventions on changes in body composition and health-related biomarkers. METHODS: A total of 41 overweight and sedentary female and male participants (38.0 ± 13 years, 90.1 ± 15 kg, 31.6 ± 6.6 kg/m2, 36.9 ± 7% fat) took part in a randomized, double-blind, and placebo-controlled study. The participants underwent baseline assessments and were then assigned to ingest 3 g/day of a placebo (PLA) or Momstamin 2-F while initiating the exercise and weight-loss program. Follow-up tests were performed after 6 and 12 weeks. Data were analyzed using general linear model statistics with repeated measures and mean changes from baseline values with 95% confidence intervals (CIs). RESULTS: No group × time × sex interaction effects were observed, so group × time effects are reported. Participants in both groups saw comparable reductions in weight. However, those with 2-FL demonstrated a significantly greater reduction in the percentage of body fat and less loss of the fat-free mass. Additionally, there was evidence that 2-FL supplementation promoted more favorable changes in resting fat oxidation, peak aerobic capacity, IL-4, and platelet aggregation, with some minimal effects on the fermentation of short-chain fatty acids and monosaccharides in fecal samples. Moreover, participants perceptions regarding some aspects of the functional capacity and ratings of the quality of life were improved, and the supplementation protocol was well tolerated, although a small, but significant, decrease in BMC was observed. CONCLUSIONS: The results support contentions that dietary supplementation of 2-FL (3 g/d) can promote fat loss and improve exercise- and diet-related markers of health and fitness in overweight sedentary individuals initiating an exercise and weight-loss program. Further research is needed to explore the potential health benefits of 2-FL supplementation in both healthy and elderly individuals (Registered clinical trial #NCT06547801)

Rearranged T Cell Receptor Sequences in the Germline Genome of Channel Catfish Are Preferentially Expressed in Response to Infection

Rearranged V(D)J genes coding for T cell receptor α and β chains are integrated into the germline genome of channel catfish. Previous analysis of expressed TCR Vβ2 repertoires demonstrated that channel catfish express multiple public clonotypes, which were shared among all the fish, following infection with a common protozoan parasite. In each case a single DNA sequence was predominately used to code for a public clonotype. We show here that the rearranged VDJ genes coding for these expressed public Vβ2 clonotypes can be amplified by PCR from germline DNA isolated from oocytes and erythrocytes. Sequencing of the Vβ2 PCR products confirmed that these expressed public Vβ2 clonotypes are integrated into the germline. Moreover, sequencing of PCR products confirmed that all five Vβ gene families and Vα1 have rearranged V(D)J genes with diverse CDR3 sequences integrated into the germline. Germline rearranged Vβ2 and Vβ4 genes retain the intron between the leader and Vβ sequence. This suggests that the germline rearranged TCR Vβ genes arose through VDJ rearrangement in T cells, and subsequently moved into the germline through DNA transposon mediated transposition. These results reveal a new dimension to the adaptive immune system of vertebrates, namely: the expression of evolutionarily conserved, rearranged V(D)J genes from the germline

Bay Breeze Influence on Surface Ozone at Edgewood, MD During July 2011

Surface ozone (O3) was analyzed to investigate the role of the bay breeze on air quality at two locations in Edgewood, Maryland (lat: 39.4deg, lon: 76.3deg) for the month of July 2011. Measurements were taken as part of the first year of NASA's "Deriving Information on Surface Conditions from Column and Vertically Resolved Observations Relevant to Air Quality" (DISCOVER-AQ) Earth Venture campaign and as part of NASA's Geostationary for Coastal and Air Pollution Events Chesapeake Bay Oceanographic campaign with DISCOVER-AQ (Geo-CAPE CBODAQ). Geo-CAPE CBODAQ complements DISCOVER-AQ by providing ship-based observations over the Chesapeake Bay. A major goal of DISCOVER-AQ is determining the relative roles of sources, photochemistry and local meteorology during air quality events in the Mid-Atlantic region of the U.S. Surface characteristics, transport and vertical structures of O3 during bay breezes were identified using in-situ surface, balloon and aircraft data, along with remote sensing equipment. Localized late day peaks in O3 were observed during bay breeze days, maximizing an average of 3 h later compared to days without bay breezes. Of the 10 days of July 2011 that violated the U.S. Environmental Protection Agency (EPA) 8 h O3 standard of 75 parts per billion by volume (ppbv) at Edgewood, eight exhibited evidence of a bay breeze circulation. The results indicate that while bay breezes and the processes associated with them are not necessary to cause exceedances in this area, bay breezes exacerbate poor air quality that sustains into the late evening hours at Edgewood. The vertical and horizontal distributions of O3 from the coastal Edgewood area to the bay also show large gradients that are often determined by boundary layer stability. Thus, developing air quality models that can sufficiently resolve these dynamics and associated chemistry, along with more consistent monitoring of O3 and meteorology on and along the complex coastline of Chesapeake Bay must be a high priority