Self-Supervised Reinforcement Learning that Transfers using Random Features

Model-free reinforcement learning algorithms have exhibited great potential in solving single-task sequential decision-making problems with high-dimensional observations and long horizons, but are known to be hard to generalize across tasks. Model-based RL, on the other hand, learns task-agnostic models of the world that naturally enables transfer across different reward functions, but struggles to scale to complex environments due to the compounding error. To get the best of both worlds, we propose a self-supervised reinforcement learning method that enables the transfer of behaviors across tasks with different rewards, while circumventing the challenges of model-based RL. In particular, we show self-supervised pre-training of model-free reinforcement learning with a number of random features as rewards allows implicit modeling of long-horizon environment dynamics. Then, planning techniques like model-predictive control using these implicit models enable fast adaptation to problems with new reward functions. Our method is self-supervised in that it can be trained on offline datasets without reward labels, but can then be quickly deployed on new tasks. We validate that our proposed method enables transfer across tasks on a variety of manipulation and locomotion domains in simulation, opening the door to generalist decision-making agents

WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens

World models play a crucial role in understanding and predicting the dynamics of the world, which is essential for video generation. However, existing world models are confined to specific scenarios such as gaming or driving, limiting their ability to capture the complexity of general world dynamic environments. Therefore, we introduce WorldDreamer, a pioneering world model to foster a comprehensive comprehension of general world physics and motions, which significantly enhances the capabilities of video generation. Drawing inspiration from the success of large language models, WorldDreamer frames world modeling as an unsupervised visual sequence modeling challenge. This is achieved by mapping visual inputs to discrete tokens and predicting the masked ones. During this process, we incorporate multi-modal prompts to facilitate interaction within the world model. Our experiments show that WorldDreamer excels in generating videos across different scenarios, including natural scenes and driving environments. WorldDreamer showcases versatility in executing tasks such as text-to-video conversion, image-tovideo synthesis, and video editing. These results underscore WorldDreamer's effectiveness in capturing dynamic elements within diverse general world environments.Comment: project page: https://world-dreamer.github.io

Alternate erosion and deposition in the Yangtze Estuary and its future change

The morphological changing trend of the Yangtze Estuary, the largest estuary of Asia, has become a focus of research in recent years. Based on a long series of topographic data from 1950 to 2015, this paper studied the erosion-deposition pattern of the entire Yangtze Estuary. An alternation between erosion and deposition was found during the past 65 years, which was in correspondence to the alternation between flood and dry periods identified by multi-year average duration days of high-level water flow (defined as discharge ≥ 60,000 m3/s, namely, D≥60,000) from the Yangtze River Basin. A quantitative relationship was further developed between the erosional/depositional rate of the Yangtze Estuary and the interpreting variables of yearly water discharge, D≥60,000 and yearly river sediment load, with contributing rates of 1%, 59% and 40%, respectively. Mechanism behind the alternate erosion and deposition pattern was analyzed by examining residual water surface slope and the corresponding capacity of sediment transport in flood and dry periods. In flood periods, a larger discharge results in steeper slope of residual water level which permits a greater capacity of sediment transport. Therefore, more bed materials can be washed to the sea, leading to erosion of the estuary. In contrast, flatter slope of residual water level occurs in dry periods, and deposition dominates the estuarine area due to the decreased capacity of sediment transport and the increased backwater effect of flood-tide. Coastal dynamics and estuarine engineering projects alter the local morphological changes, but slightly affect the total erosional/depositional rate of the whole estuarine region. Heavy sedimentation within the Yangtze Estuary after the impoundment of the Three Gorges Dam can be attributed to the reduced occurrence frequency of flood years due to water regulation by the dam, and largely (at least 36%–52%) sourced from the sea. Deposition is still possible to occur in the Yangtze Estuary in the future, because the multi-year average D≥60,000 is unlikely to exceed the critical value of 14 days/yr which corresponds to the future equilibrium state of the Yangtze Estuary, under the water regulation of the large cascade dams in the upper Yangtze. Nevertheless, the mean depositional rate will not surpass the peak value of the past years, since the total sediment load entering the Yangtze Estuary has presented a decreasing trend

Impact of water sediment diversion and afflux on erosion deposition in the Luoshan Hankou reach, 1 middle Yangtze River, China

It is not yet fully understood how water-sediment diversion and afflux along a mainstream reach of a river affect erosion-deposition in downstream reaches. This study focuses on the Luoshan-Hankou mainstream reach of the middle Yangtze River, China. The Luoshan-Hankou reach is vitally important for flood control, being located downstream of three diversion mouths and an afflux outlet along the Jingjiang reach. We establish empirical formulae for sediment transport rates at boundary cross-sections, and hence estimate the amount and proportion of erosion-deposition and its relative increase (termed erosion-deposition promotion) in the Luoshan-Hankou reach. We then propose critical net water supplies from Dongting Lake to Luoshan-Hankou reach based on maxima and equilibria of erosion-deposition and its promotion. It is found that net water supply partly drives erosion-deposition in the Luoshan-Hankou reach where maximal proportions of deposition and deposition-promotion may be approximated by 0.01c-37.67 and 0.01c-37.67 + c-1, in which c is a dimensionless parameter representing the erosion-deposition condition in Luoshan-Hankou reach for no water-sediment exchange. At Zhicheng hydrological station, the critical ratio of net water supply to overall water discharge is 0.418c-33.33-1, and critical net water supply ratios for equilibria of erosion-deposition and its promotion are −1 (or c-33.33-1) and 0 (or (0.06 + 3.257c54.61)-1-1). A chart based on net water supply and c is devised representing four types of erosion–deposition and its promotion for the Luoshan-Hankou reach. Historical data over the past 65 years demonstrate that erosion-deposition and its promotion in the reach are respectively governed by c and net water supply; there is a remarkable shift from alternate erosion-deposition to monotonic erosion whilst the erosion-deposition effect remains consistent. The foregoing are in agreement with observed data, and comparable with data for the Jingjiang reach (affected by the three water-sediment diversion mouths). Satisfactory flood-control conditions in the convergence zone between the Yangtze mainstream and Dongting Lake accompanied by increasing erosion in the Luoshan-Hankou reach are predicted for the future

Traffic condition tracking and visualization in virtual city testbed

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2011.Cataloged from PDF version of thesis.Includes bibliographical references (p. 31).Computer traffic simulation is a tool widely used to understand how humans behave under varying traffic conditions. The Virtual City Testbed is a traffic simulation framework built to closely model human behavior by allowing direct user interaction in the simulation. Using the testbed, human subjects can remotely control vehicles in the virtual environment. A virtual positioning system (VPS) is displayed with the testbed client. It tracks and visualizes traffic, disruptions, and tolls local to the user's position. As traffic conditions shift, the VPS dynamically updates to reflect the changes. Together, the testbed and VPS provide an environment for studying how traffic conditions affect decision making.by Boyuan Zhu.M.Eng

Constrained tropical land temperature-precipitation sensitivity reveals decreasing evapotranspiration and faster vegetation greening in CMIP6 3 projections

AbstractOver the tropical land surface, accurate estimates of future changes in temperature, precipitation and evapotranspiration are crucial for ecological sustainability, but remain highly uncertain. Here we develop a series of emergent constraints (ECs) by using historical and future outputs from the Coupled Model Inter-comparison Project Phase 6 (CMIP6) Earth System Models under the four basic Shared Socio-economic Pathway scenarios (SSP126, SSP245, SSP370, and SSP585). Results show that the temperature sensitivity to precipitation during 2015–2100, which varies substantially in the original CMIP6 outputs, becomes systematically negative across SSPs after application of the EC, with absolute values between −1.10 °C mm−1 day and −3.52 °C mm−1 day, and with uncertainties reduced by 9.4% to 41.4%. The trend in tropical land-surface evapotranspiration, which was increasing by 0.292 mm yr−1 in the original CMIP6 model outputs, becomes significantly negative (−0.469 mm yr−1) after applying the constraint. Moreover, we find a significant increase of 58.7% in the leaf area index growth rate.</jats:p