Self-Supervised Reinforcement Learning that Transfers using Random Features

Model-free reinforcement learning algorithms have exhibited great potential in solving single-task sequential decision-making problems with high-dimensional observations and long horizons, but are known to be hard to generalize across tasks. Model-based RL, on the other hand, learns task-agnostic models of the world that naturally enables transfer across different reward functions, but struggles to scale to complex environments due to the compounding error. To get the best of both worlds, we propose a self-supervised reinforcement learning method that enables the transfer of behaviors across tasks with different rewards, while circumventing the challenges of model-based RL. In particular, we show self-supervised pre-training of model-free reinforcement learning with a number of random features as rewards allows implicit modeling of long-horizon environment dynamics. Then, planning techniques like model-predictive control using these implicit models enable fast adaptation to problems with new reward functions. Our method is self-supervised in that it can be trained on offline datasets without reward labels, but can then be quickly deployed on new tasks. We validate that our proposed method enables transfer across tasks on a variety of manipulation and locomotion domains in simulation, opening the door to generalist decision-making agents

Alternate erosion and deposition in the Yangtze Estuary and its future change

The morphological changing trend of the Yangtze Estuary, the largest estuary of Asia, has become a focus of research in recent years. Based on a long series of topographic data from 1950 to 2015, this paper studied the erosion-deposition pattern of the entire Yangtze Estuary. An alternation between erosion and deposition was found during the past 65 years, which was in correspondence to the alternation between flood and dry periods identified by multi-year average duration days of high-level water flow (defined as discharge ≥ 60,000 m3/s, namely, D≥60,000) from the Yangtze River Basin. A quantitative relationship was further developed between the erosional/depositional rate of the Yangtze Estuary and the interpreting variables of yearly water discharge, D≥60,000 and yearly river sediment load, with contributing rates of 1%, 59% and 40%, respectively. Mechanism behind the alternate erosion and deposition pattern was analyzed by examining residual water surface slope and the corresponding capacity of sediment transport in flood and dry periods. In flood periods, a larger discharge results in steeper slope of residual water level which permits a greater capacity of sediment transport. Therefore, more bed materials can be washed to the sea, leading to erosion of the estuary. In contrast, flatter slope of residual water level occurs in dry periods, and deposition dominates the estuarine area due to the decreased capacity of sediment transport and the increased backwater effect of flood-tide. Coastal dynamics and estuarine engineering projects alter the local morphological changes, but slightly affect the total erosional/depositional rate of the whole estuarine region. Heavy sedimentation within the Yangtze Estuary after the impoundment of the Three Gorges Dam can be attributed to the reduced occurrence frequency of flood years due to water regulation by the dam, and largely (at least 36%–52%) sourced from the sea. Deposition is still possible to occur in the Yangtze Estuary in the future, because the multi-year average D≥60,000 is unlikely to exceed the critical value of 14 days/yr which corresponds to the future equilibrium state of the Yangtze Estuary, under the water regulation of the large cascade dams in the upper Yangtze. Nevertheless, the mean depositional rate will not surpass the peak value of the past years, since the total sediment load entering the Yangtze Estuary has presented a decreasing trend

Traffic condition tracking and visualization in virtual city testbed

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2011.Cataloged from PDF version of thesis.Includes bibliographical references (p. 31).Computer traffic simulation is a tool widely used to understand how humans behave under varying traffic conditions. The Virtual City Testbed is a traffic simulation framework built to closely model human behavior by allowing direct user interaction in the simulation. Using the testbed, human subjects can remotely control vehicles in the virtual environment. A virtual positioning system (VPS) is displayed with the testbed client. It tracks and visualizes traffic, disruptions, and tolls local to the user's position. As traffic conditions shift, the VPS dynamically updates to reflect the changes. Together, the testbed and VPS provide an environment for studying how traffic conditions affect decision making.by Boyuan Zhu.M.Eng

Constrained tropical land temperature-precipitation sensitivity reveals decreasing evapotranspiration and faster vegetation greening in CMIP6 3 projections

AbstractOver the tropical land surface, accurate estimates of future changes in temperature, precipitation and evapotranspiration are crucial for ecological sustainability, but remain highly uncertain. Here we develop a series of emergent constraints (ECs) by using historical and future outputs from the Coupled Model Inter-comparison Project Phase 6 (CMIP6) Earth System Models under the four basic Shared Socio-economic Pathway scenarios (SSP126, SSP245, SSP370, and SSP585). Results show that the temperature sensitivity to precipitation during 2015–2100, which varies substantially in the original CMIP6 outputs, becomes systematically negative across SSPs after application of the EC, with absolute values between −1.10 °C mm−1 day and −3.52 °C mm−1 day, and with uncertainties reduced by 9.4% to 41.4%. The trend in tropical land-surface evapotranspiration, which was increasing by 0.292 mm yr−1 in the original CMIP6 model outputs, becomes significantly negative (−0.469 mm yr−1) after applying the constraint. Moreover, we find a significant increase of 58.7% in the leaf area index growth rate.</jats:p

Culture Change and Affectionate Communication in China and the United States: Evidence From Google Digitized Books 1960–2008

Humans are born with the ability and the need for affection, but communicating affection as a social behavior is historically bound. Based on the digitized books of Google Ngram Viewer from 1960 through 2008, the present research investigated affectionate communication (AC) in China and the United States, and its changing landscape along with social changes from collectivist to individualistic environments. In particular, we analyzed the frequency in terms of verbal affection (e.g., love you, like you), non-verbal affection (e.g., hug, kiss), and individualism (indicated by the use of first-person singular pronouns such as I, me, and myself) in Chinese and American books. The results revealed an increasing trend for AC in recent decades, although the frequency of affection words was lower in Chinese than in American books. Further, individualism was positively related to the frequency of affection words in both Chinese and American books. These results demonstrate the effect of cultural changes on AC, in that affection exchange becomes popular in adaptation to individualistic urban environments. These findings exemplify a cross-cultural difference in the expression of love and the cultural universality of social change in Eastern and Western societies