A comparison of reinforcement learning algorithms in fairness-oriented OFDMA schedulers

Due to large-scale control problems in 5G access networks, the complexity of radioresource management is expected to increase significantly. Reinforcement learning is seen as apromising solution that can enable intelligent decision-making and reduce the complexity of differentoptimization problems for radio resource management. The packet scheduler is an importantentity of radio resource management that allocates users’ data packets in the frequency domainaccording to the implemented scheduling rule. In this context, by making use of reinforcementlearning, we could actually determine, in each state, the most suitable scheduling rule to be employedthat could improve the quality of service provisioning. In this paper, we propose a reinforcementlearning-based framework to solve scheduling problems with the main focus on meeting the userfairness requirements. This framework makes use of feed forward neural networks to map momentarystates to proper parameterization decisions for the proportional fair scheduler. The simulation resultsshow that our reinforcement learning framework outperforms the conventional adaptive schedulersoriented on fairness objective. Discussions are also raised to determine the best reinforcement learningalgorithm to be implemented in the proposed framework based on various scheduler settings

Towards 5G: A reinforcement learning-based scheduling solution for data traffic management

Dominated by delay-sensitive and massive data applications, radio resource management in 5G access networks is expected to satisfy very stringent delay and packet loss requirements. In this context, the packet scheduler plays a central role by allocating user data packets in the frequency domain at each predefined time interval. Standard scheduling rules are known limited in satisfying higher quality of service (QoS) demands when facing unpredictable network conditions and dynamic traffic circumstances. This paper proposes an innovative scheduling framework able to select different scheduling rules according to instantaneous scheduler states in order to minimize the packet delays and packet drop rates for strict QoS requirements applications. To deal with real-time scheduling, the reinforcement learning (RL) principles are used to map the scheduling rules to each state and to learn when to apply each. Additionally, neural networks are used as function approximation to cope with the RL complexity and very large representations of the scheduler state space. Simulation results demonstrate that the proposed framework outperforms the conventional scheduling strategies in terms of delay and packet drop rate requirements

Enhancing user fairness in OFDMA radio access networks through machine learning

The problem of radio resource scheduling subject to fairness satisfaction is very challenging even in future radio access networks. Standard fairness criteria aim to find the best trade-off between overall throughput maximization and user fairness satisfaction under various types of network conditions. However, at the Radio Resource Management (RRM) level, the existing schedulers are rather static being unable to react according to the momentary networking conditions so that the user fairness measure is maximized all time. This paper proposes a dynamic scheduler framework able to parameterize the proportional fair scheduling rule at each Transmission Time Interval (TTI) to improve the user fairness. To deal with the framework complexity, the parameterization decisions are approximated by using the neural networks as non-linear functions. The actor-critic Reinforcement Learning (RL) algorithm is used to learn the best set of non-linear functions that approximate the best fairness parameters to be applied in each momentary state. Simulations results reveal that the proposed framework outperforms the existing fairness adaptation techniques as well as other types of RL-based schedulers

Comment on "Giant absorption cross section of ultracold neutrons in Gadolinium"

Rauch et al (PRL 83, 4955, 1999) have compared their measurements of the Gd cross section for Ultra-cold neutrons with an exptrapolation of the cross section for thermal neutrons and interpreted the discrepancy in terms of coherence properties of the neutron. We show the extrapolation used is based on a misunderstanding and that coherence properties play no role in absorption.Comment: 2 pages, 1 postscript figure, comment on Rauch et al, PRL 83,4955 (1999

Local management and landscape composition affect predatory mites in European wine-growing regions

Sustainable land use in agricultural landscapes is essential to counteract the global decline of biodiversity, as well to ensure ecosystem services like natural pest control. Phytoseiid mites are key natural enemies of pest mites in vineyards but how local management and landscape context affect phytoseiid mites remains poorly known. In this study, we examined the effects of farming systems, inter-row management and landscape composition on phytoseiid mite communities in 156 vineyards across five European wine-growing regions. Our results showed that phytoseiid communities were mainly dominated by one or two phytoseiid species across Europe and that local management was a major factor affecting population densities. According to the wine-growing regions, phytoseiid mite densities benefited from integrated pest management or conventional farming compared to organic farming and from spontaneous vegetation cover compared to seeded cover crops. Moreover, mite densities benefited from increasing proportions of vineyards at the landscape scale. The farming systems effects were most likely related to the positive impact of the lower pesticide use in integrated and conventional vineyards. The positive effect of spontaneous vegetation cover could be related to a better supply of nutritive pollen as food resource compared to seeded cover crops, which depends on the plant species in the inter-row. Our findings indicated accordingly that a reduced pesticide use, and inter-row management are crucial factors for promoting pest control by predatory mites in European vineyards. Moreover, the proportion of viticultural area in the landscape is a considerable factor to retain stable phytoseiid mite populations.This research was funded by the research project SECBIVIT, which was funded through the 2017-2018 Belmont Forum and BiodivERsA joint call for research proposals, under the BiodivScen ERA-Net COFUND program, with the funding organizations: Agencia Estatal de Investigación (Ministerio de ciencia e innovación/ES/Grant #10.13039/501100011033), Austrian Science Fund (AT/Grant #I 4025-B32), Federal Ministry of Education and Research and Projektträger VDI/VDE Innovation + Technik GmbH (DE), French National Research Agency (FR), Netherlands Organisation for Scientific Research (NL), National Science Foundation (US/Grant #1850943) and Romanian Executive Agency for Higher Education, Research, Development and Innovation Funding (RO)

High-dimensional quantum dynamics of adsorption and desorption of H2_2 at Cu(111)

We performed high-dimensional quantum dynamical calculations of the dissociative adsorption and associative desorption of hydrogen on Cu(111). The potential energy surface (PES) is obtained from density functional theory calculations. Two regimes of dynamics are found, at low energies sticking is determined by the minimum energy barrier, at high energies by the distribution of barrier heights. Experimental results are well-reproduced qualitatively, but some quantitative discrepancies are identified as well.Comment: 4 two column pages, revtex, 4 figures, to appear in Phys. Rev. Let

Insights into the Second Law of Thermodynamics from Anisotropic Gas-Surface Interactions

Thermodynamic implications of anisotropic gas-surface interactions in a closed molecular flow cavity are examined. Anisotropy at the microscopic scale, such as might be caused by reduced-dimensionality surfaces, is shown to lead to reversibility at the macroscopic scale. The possibility of a self-sustaining nonequilibrium stationary state induced by surface anisotropy is demonstrated that simultaneously satisfies flux balance, conservation of momentum, and conservation of energy. Conversely, it is also shown that the second law of thermodynamics prohibits anisotropic gas-surface interactions in "equilibrium", even for reduced dimensionality surfaces. This is particularly startling because reduced dimensionality surfaces are known to exhibit a plethora of anisotropic properties. That gas-surface interactions would be excluded from these anisotropic properties is completely counterintuitive from a causality perspective. These results provide intriguing insights into the second law of thermodynamics and its relation to gas-surface interaction physics.Comment: 28 pages, 11 figure

Potential Energy Surface for H_2 Dissociation over Pd(100)

The potential energy surface (PES) of dissociative adsorption of H_2 on Pd(100) is investigated using density functional theory and the full-potential linear augmented plane wave (FP-LAPW) method. Several dissociation pathways are identified which have a vanishing energy barrier. A pronounced dependence of the potential energy on ``cartwheel'' rotations of the molecular axis is found. The calculated PES shows no indication of the presence of a precursor state in front of the surface. Both results indicate that steering effects determine the observed decrease of the sticking coefficient at low energies of the H_2 molecules. We show that the topology of the PES is related to the dependence of the covalent H(s)-Pd(d) interactions on the orientation of the H_2 molecule.Comment: RevTeX, 8 pages, 5 figures in uufiles forma

Decoherence, fluctuations and Wigner function in neutron optics

We analyze the coherence properties of neutron wave packets, after they have interacted with a phase shifter undergoing different kinds of statistical fluctuations. We give a quantitative (and operational) definition of decoherence and compare it to the standard deviation of the distribution of the phase shifts. We find that in some cases the neutron ensemble is more coherent, even though it has interacted with a wider (i.e. more disordered) distribution of shifts. This feature is independent of the particular definition of decoherence: this is shown by proposing and discussing an alternative definition, based on the Wigner function, that displays a similar behavior. We briefly discuss the notion of entropy of the shifts and find that, in general, it does not correspond to that of decoherence of the neutron.Comment: 18 pages, 7 figure