Model-Based Reinforcement Learning with Isolated Imaginations

World models learn the consequences of actions in vision-based interactive systems. However, in practical scenarios like autonomous driving, noncontrollable dynamics that are independent or sparsely dependent on action signals often exist, making it challenging to learn effective world models. To address this issue, we propose Iso-Dream++, a model-based reinforcement learning approach that has two main contributions. First, we optimize the inverse dynamics to encourage the world model to isolate controllable state transitions from the mixed spatiotemporal variations of the environment. Second, we perform policy optimization based on the decoupled latent imaginations, where we roll out noncontrollable states into the future and adaptively associate them with the current controllable state. This enables long-horizon visuomotor control tasks to benefit from isolating mixed dynamics sources in the wild, such as self-driving cars that can anticipate the movement of other vehicles, thereby avoiding potential risks. On top of our previous work, we further consider the sparse dependencies between controllable and noncontrollable states, address the training collapse problem of state decoupling, and validate our approach in transfer learning setups. Our empirical study demonstrates that Iso-Dream++ outperforms existing reinforcement learning models significantly on CARLA and DeepMind Control.Comment: arXiv admin note: substantial text overlap with arXiv:2205.1381

Efficacy and safety of a combination of miglitol, metformin and insulin aspart in the treatment of type 2 diabetes

Purpose: To study the clinical effect of combining insulin aspart with different drugs in the treatment oftype 2 diabetes mellitus (T2DM).Methods: Two hundred and thirty-seven T2DM patients admitted to the Endocrinology Department of the Second Affiliated Hospital of Kunming Medical University from March to September 2018 were selected as subjects in this study. Miglitol and metformin were used in combination with insulin aspart in the treatment of T2DM. In addition, data on the effectiveness and safety of different treatment options,such as patient’s weight, waist circumference, blood glucose indicators, indices of heart, liver and kidney functions, and incidence of complications were recorded and compared between the two groups.Results: The use of a combination of miglitol and insulin aspart produced an excellent hypoglycaemic effect, and it significantly reduced the incidence of sensory neuropathy in the eyes and distal limbs (p < 0.05). The use of combination of metformin and insulin aspart effectively protected the heart and kidney, and prevented hypoglycaemia (p < 0.05).Conclusion: These results suggest that treatment with a combination of miglitol and insulin aspart is suitable for patients with T2DM whose blood sugar levels are out of control, while combined treatment with metformin and insulin aspart is more suited for patients who desire to reduce blood sugar and blood lipids through weight loss, and patients with cardiac and renal insufficiency

Equilibrium responses of global net primary production and carbon storage to doubled atmospheric carbon dioxide: sensitivity to changes in vegetation nitrogen concentration

We ran the terrestrial ecosystem model (TEM) for the globe at 0.5° resolution for atmospheric CO2 concentrations of 340 and 680 parts per million by volume (ppmv) to evaluate global and regional responses of net primary production (NPP) and carbon storage to elevated CO2 for their sensitivity to changes in vegetation nitrogen concentration. At 340 ppmv, TEM estimated global NPP of 49.0 1015 g (Pg) C yr−1 and global total carbon storage of 1701.8 Pg C; the estimate of total carbon storage does not include the carbon content of inert soil organic matter. For the reference simulation in which doubled atmospheric CO2 was accompanied with no change in vegetation nitrogen concentration, global NPP increased 4.1 Pg C yr−1 (8.3%), and global total carbon storage increased 114.2 Pg C. To examine sensitivity in the global responses of NPP and carbon storage to decreases in the nitrogen concentration of vegetation, we compared doubled CO2 responses of the reference TEM to simulations in which the vegetation nitrogen concentration was reduced without influencing decomposition dynamics (“lower N” simulations) and to simulations in which reductions in vegetation nitrogen concentration influence decomposition dynamics (“lower N+D” simulations). We conducted three lower N simulations and three lower N+D simulations in which we reduced the nitrogen concentration of vegetation by 7.5, 15.0, and 22.5%. In the lower N simulations, the response of global NPP to doubled atmospheric CO2 increased approximately 2 Pg C yr−1 for each incremental 7.5% reduction in vegetation nitrogen concentration, and vegetation carbon increased approximately an additional 40 Pg C, and soil carbon increased an additional 30 Pg C, for a total carbon storage increase of approximately 70 Pg C. In the lower N+D simulations, the responses of NPP and vegetation carbon storage were relatively insensitive to differences in the reduction of nitrogen concentration, but soil carbon storage showed a large change. The insensitivity of NPP in the N+D simulations occurred because potential enhancements in NPP associated with reduced vegetation nitrogen concentration were approximately offset by lower nitrogen availability associated with the decomposition dynamics of reduced litter nitrogen concentration. For each 7.5% reduction in vegetation nitrogen concentration, soil carbon increased approximately an additional 60 Pg C, while vegetation carbon storage increased by only approximately 5 Pg C. As the reduction in vegetation nitrogen concentration gets greater in the lower N+D simulations, more of the additional carbon storage tends to become concentrated in the north temperate-boreal region in comparison to the tropics. Other studies with TEM show that elevated CO2 more than offsets the effects of climate change to cause increased carbon storage. The results of this study indicate that carbon storage would be enhanced by the influence of changes in plant nitrogen concentration on carbon assimilation and decomposition rates. Thus changes in vegetation nitrogen concentration may have important implications for the ability of the terrestrial biosphere to mitigate increases in the atmospheric concentration of CO2 and climate changes associated with the increases

Regional and global elevational patterns of microbial species richness and evenness

8 páginas, 4 figurasAlthough elevational gradients in microbial biodiversity have attracted increasing attention recently, the generality in the patterns and underlying mechanisms are still poorly resolved. Further, previous studies focused mostly on species richness, while left understudied evenness, another important aspect of biodiversity. Here, we studied the elevational patterns in species richness and evenness of stream biofi lm bacteria and diatoms in six mountains in Asia and Europe. We also reviewed published results for elevational richness patterns for soil and stream microbes in a literature analysis. Our results revealed that even within the same ecosystem type (that is, stream) or geographical region, bacteria and diatoms showed contrasting patterns in diversity. Stream microbes, including present stream data, tend to show signifi cantly increasing or decreasing elevational patterns in richness, contrasting the fi ndings for soil microbes that typically showed nonsignifi cant or signifi cantly decreasing patterns. In all six mountains for bacteria and in four mountains for diatoms, species richness and evenness were positively correlated. Th e variation in bacteria and diatom richness and evenness were substantially explained by anthropogenic driven factors, such as total phosphorus (TP). However, diatom richness and evenness were also related to diff erent main drivers as richness was mostly related to pH, while evenness was most explained by TP. Our results highlight the lack of consistent elevational biodiversity patterns of microbes and further indicate that the two facets of biodiversity may respond diff erently to environmental gradients.JW was supported by NSFC grant 41273088, 41571058, 40903031 and CAS oversea visiting scholarship (2011-115). JS and JW were supported by Emil Aaltonen Foundation. JS and JW were supported by 973 Program (2012CB956100). Th e fi eld trips were partly supported by Air and Water Conservation Fund (GEFC12-14, National Geography of Science) to JW, and DISPERSAL 829/2013 from the Spanish National Parks Research Programme OAPNMAGRAMA to EOC.Peer reviewe

On the opportunistic connectivity of large-scale urban vehicular networks

Understanding and characterizing the properties of vehicular networks has become hugely important because of their wide applications and fast development. Due to the unique property of vehicular networks that nodes change their locations at high speed in anytime, their connectivity is an important and unique property. In this poster, based on realistic vehicular mobility traces, we reveal the opportunistic connectivity property of large-scale urban vehicular networks. Moreover, through analysis, we unveil the fundamental relationships and tradeoffs between the opportunistic connectivity and network parameters in terms of data size, delivery delay, transmission energy, etc. © 2013 IEEE

Isolating and Leveraging Controllable and Noncontrollable Visual Dynamics in World Models

World models learn the consequences of actions in vision-based interactive systems. However, in practical scenarios such as autonomous driving, there commonly exists noncontrollable dynamics independent of the action signals, making it difficult to learn effective world models. To tackle this problem, we present a novel reinforcement learning approach named Iso-Dream, which improves the Dream-to-Control framework in two aspects. First, by optimizing the inverse dynamics, we encourage the world model to learn controllable and noncontrollable sources of spatiotemporal changes on isolated state transition branches. Second, we optimize the behavior of the agent on the decoupled latent imaginations of the world model. Specifically, to estimate state values, we roll-out the noncontrollable states into the future and associate them with the current controllable state. In this way, the isolation of dynamics sources can greatly benefit long-horizon decision-making of the agent, such as a self-driving car that can avoid potential risks by anticipating the movement of other vehicles. Experiments show that Iso-Dream is effective in decoupling the mixed dynamics and remarkably outperforms existing approaches in a wide range of visual control and prediction domains

Enhancing PAPR and Throughput for DFT-s-OFDM System Using FTN and IOTA Filtering

High frequency wireless communication aims to provide ultra high-speed transmissions for various application scenarios. The waveform design for high frequency communication is challenging due to the requirements for high spectrum efficiency, as well as good hardware compatibility. With high flexibility and low peak-to-average power ratio (PAPR), discrete Fourier transformation spreading-based orthogonal frequency division multiplexing (DFT-s-OFDM) can be a promising candidate waveform. To further enhance the spectral efficiency, we integrate faster-than-Nyquist (FTN) signaling in DFT-s-OFDM, and find that the PAPR performance can also be improved. While FTN can introduce increased inter-symbol interference (ISI), in this paper, we deploy an isotropic orthogonal transform algorithm (IOTA) filter for FTN-enhanced DFT-s-OFDM, where the compact time-frequency structure of the IOTA filter can significantly reduce the ISI. Simulation results show that the proposed waveform is capable of achieving good performance in PAPR, bit error rate (BER) and throughput, simultaneously, with 3.5 dB gain in PAPR and 50% gain in throughput

Dynamic Speed Control of Unmanned Aerial Vehicles for Data Collection under Internet of Things

With the new advancements in flight control and integrated circuit (IC) technology, unmanned aerial vehicles (UAVs) have been widely used in various applications. One of the typical application scenarios is data collection for large-scale and remote sensor devices in the Internet of things (IoT). However, due to the characteristics of massive connections, access collisions in the MAC layer lead to high power consumption for both sensor devices and UAVs, and low efficiency for the data collection. In this paper, a dynamic speed control algorithm for UAVs (DSC-UAV) is proposed to maximize the data collection efficiency, while alleviating the access congestion for the UAV-based base stations. With a cellular network considered for support of the communication between sensor devices and drones, the connection establishment process was analyzed and modeled in detail. In addition, the data collection efficiency is also defined and derived. Based on the analytical models, optimal speed under different sensor device densities is obtained and verified. UAVs can dynamically adjust the speed according to the sensor device density under their coverages to keep high data collection efficiency. Finally, simulation results are also conducted to verify the accuracy of the proposed analytical models and show that the DSC-UAV outperforms others with the highest data collection efficiency, while maintaining a high successful access probability, low average access delay, low block probability, and low collision probability

Effect of Application Rates of N and P Fertilizers on Soil Nematode Community Structure in Mollisols

Long-term application of chemical fertilizer poses an environmental threat to belowground ecosystems. However, the impact of nitrogen (N) or phosphorus (P) fertilizers on soil biodiversity and the conditions of soil food web remains largely unknown. Soil nematodes are the most abundant multicellular soil animals and serve as excellent bioindicators of soil. Here, we investigated soil nematode communities and food web structure in a long-term experiment with different application rates of N and P fertilizers in northeast China. The application of N and P fertilizers increased the abundance of bacterivores but suppressed the abundance of omnivores and predators. The abundance of bacterivores exhibited an increasing trend, while that of omnivores and predators showed a decreasing trend with increasing rates of N and P fertilizers. Plant parasites displayed a decreasing trend in response to N fertilizer, but not to P fertilizer. N and P fertilizers also altered nematode functional guild composition, with N fertilizer increasing the abundance of Ba1, and P fertilizer increasing the abundance of Fu2 and Ba3. Nonmetric multidimensional scaling (NMDS) analysis revealed apparent successions of nematode communities from no fertilizer soils to high rates of N or P fertilizer soils at both the genus and functional guild levels. Furthermore, N and P fertilizers resulted in different nematode communities. In terms of nematode food web indices, N fertilizer increased the enrichment index (EI) but reduced the channel index (CI) and structure index (SI), whereas P fertilizer only reduced the SI value. High rates of N and P fertilizers increased the respired carbon of bacterivores but reduced the respired carbon of predators. Mantel tests revealed significant correlations between soil properties and the community composition of both fungivores and omnivores. Among all soil properties, available phosphorus (AP) had the greatest influence on the community structure of soil nematodes. Our findings indicate that N fertilizer has a powerful effect on nematode food web structure, while P fertilizer exerts a stronger effect on soil nematode community composition