Dynamic Weights in Multi-Objective Deep Reinforcement Learning

Many real-world decision problems are characterized by multiple conflicting objectives which must be balanced based on their relative importance. In the dynamic weights setting the relative importance changes over time and specialized algorithms that deal with such change, such as a tabular Reinforcement Learning (RL) algorithm by Natarajan and Tadepalli (2005), are required. However, this earlier work is not feasible for RL settings that necessitate the use of function approximators. We generalize across weight changes and high-dimensional inputs by proposing a multi-objective Q-network whose outputs are conditioned on the relative importance of objectives and we introduce Diverse Experience Replay (DER) to counter the inherent non-stationarity of the Dynamic Weights setting. We perform an extensive experimental evaluation and compare our methods to adapted algorithms from Deep Multi-Task/Multi-Objective Reinforcement Learning and show that our proposed network in combination with DER dominates these adapted algorithms across weight change scenarios and problem domains

Influence of Sea-Ice Anomalies on Antarctic Precipitation Using Source Attribution in the Community Earth System Model

We conduct sensitivity experiments using a general circulation model that has an explicit water source tagging capability forced by prescribed composites of pre-industrial sea-ice concentrations (SICs) and corresponding sea surface temperatures (SSTs) to understand the impact of sea-ice anomalies on regional evaporation, moisture transport and sourcereceptor relationships for Antarctic precipitation in the absence of anthropogenic forcing. Surface sensible heat fluxes, evaporation and column-integrated water vapor are larger over Southern Ocean (SO) areas with lower SICs. Changes in Antarctic precipitation and its source attribution with SICs have a strong spatial variability. Among the tagged source regions, the Southern Ocean (south of 50 S) contributes the most (40 %) to the Antarctic total precipitation, followed by more northerly ocean basins, most notably the South Pacific Ocean (27%), southern Indian Ocean (16 %) and South Atlantic Ocean (11 %). Comparing two experiments prescribed with high and low pre-industrial SICs, respectively, the annual mean Antarctic precipitation is about 150 Gt yr1 (or 6 %) more in the lower SIC case than in the higher SIC case. This difference is larger than the model-simulated interannual variability in Antarctic precipitation (99 Gt yr1). The contrast in contribution from the Southern Ocean, 102 Gt yr1, is even more significant compared to the interannual variability of 35 Gt yr1 in Antarctic precipitation that originates from the Southern Ocean. The horizontal transport pathways from individual vapor source regions to Antarctica are largely determined by large-scale atmospheric circulation patterns. Vapor from lower-latitude source regions takes elevated pathways to Antarctica. In contrast, vapor from the Southern Ocean moves southward within the lower troposphere to the Antarctic continent along moist isentropes that are largely shaped by local ambient conditions and coastal topography. This study also highlights the importance of atmospheric dynamics in affecting the thermodynamic impact of sea-ice anomalies associated with natural variability on Antarctic precipitation. Our analyses of the seasonal contrast in changes of basin-scale evaporation, moisture flux and precipitation suggest that the impact of SIC anomalies on regional Antarctic precipitation depends on dynamic changes that arise from SICSST perturbations along with internal variability. The latter appears to have a more significant effect on the moisture transport in austral winter than in summer

A new regional climate model for POLAR-CORDEX : evaluation of a 30-year hindcast with COSMO-CLM2 over Antarctica

Continent-wide climate information over the Antarctic Ice Sheet (AIS) is important to obtain accurate information of present climate and reduce uncertainties of the ice sheet mass balance response and resulting global sea level rise to future climate change. In this study, the COSMO-CLM2 Regional Climate Model is applied over the AIS and adapted for the specific meteorological and climatological conditions of the region. A 30-year hindcast was performed and evaluated against observational records consisting of long-term ground-based meteorological observations, automatic weather stations, radiosoundings, satellite records, stake measurements and ice cores. Reasonable agreement regarding the surface and upper-air climate is achieved by the COSMO-CLM2 model, comparable to the performance of other state-of-the-art climate models over the AIS. Meteorological variability of the surface climate is adequately simulated, and biases in the radiation and surface mass balance are small. The presented model therefore contributes as a new member to the COordinated Regional Downscaling EXperiment project over the AIS (POLAR-CORDEX) and the CORDEX-CORE initiative

Atomic Layer Deposition-Based Synthesis of Photoactive TiO2 Nanoparticle Chains by Using Carbon Nanotubes as Sacrificial Templates

Highly ordered and self supported anatase TiO2 nanoparticle chains were fabricated by calcining conformally TiO2 coated multi-walled carbon nanotubes (MWCNTs). During annealing, the thin tubular TiO2 coating that was deposited onto the MWCNTs by atomic layer deposition (ALD) was transformed into chains of TiO2 nanoparticles (~12 nm diameter) with an ultrahigh surface area (137 cm2 per cm2 of substrate), while at the same time the carbon from the MWCNTs was removed. Photocatalytic tests on the degradation of acetaldehyde proved that these forests of TiO2 nanoparticle chains are highly photo active under UV light because of their well crystallized anatase phase

Dynamics of Mutant Cells in Hierarchical Organized Tissues

Most tissues in multicellular organisms are maintained by continuous cell renewal processes. However, high turnover of many cells implies a large number of error-prone cell divisions. Hierarchical organized tissue structures with stem cell driven cell differentiation provide one way to prevent the accumulation of mutations, because only few stem cells are long lived. We investigate the deterministic dynamics of cells in such a hierarchical multi compartment model, where each compartment represents a certain stage of cell differentiation. The dynamics of the interacting system is described by ordinary differential equations coupled across compartments. We present analytical solutions for these equations, calculate the corresponding extinction times and compare our results to individual based stochastic simulations. Our general compartment structure can be applied to different tissues, as for example hematopoiesis, the epidermis, or colonic crypts. The solutions provide a description of the average time development of stem cell and non stem cell driven mutants and can be used to illustrate general and specific features of the dynamics of mutant cells in such hierarchically structured populations. We illustrate one possible application of this approach by discussing the origin and dynamics of PIG-A mutant clones that are found in the bloodstream of virtually every healthy adult human. From this it is apparent, that not only the occurrence of a mutant but also the compartment of origin is of importance

Significant Spatial Variability in Radar-Derived West Antarctic Accumulation Linked to Surface Winds and Topography

Across the Antarctic Ice Sheet, accumulation heavily influences firn compaction and surface height changes. Therefore, accumulation varies over short distances (25 km) that are too coarse to resolve this variability. To address this limitation, we construct a fine-scale accumulation product from airborne snow radar observations by superimposing along-track fluctuations in accumulation onto an atmospheric reanalysis product. Our resulting airborne product reflects large-scale (>25 km) orographic precipitation patterns while providing robust and unprecedented insight into Antarctic accumulation variability on subgrid scales. On these smaller scales, we find significant, regionally dependent accumulation variability ((sub relative) > 40%). This variability in accumulation is correlated with variability in topographic surface slope in the wind direction (p < 0.01), confirming that subgrid-scale accumulation variability is driven by snow redistribution by wind

Validation of Two Nonlinear System Identification Techniques Using an Experimental Testbed

The identification of a nonlinear system is performed using experimental data and two different techniques, i.e. a method based on the Wavelet transform and the Restoring Force Surface method. Both techniques exploit the system free response and result in the estimation of linear and nonlinear physical parameters