Delay Constrained Buffer-Aided Relay Selection in the Internet of Things with Decision-Assisted Reinforcement Learning

This paper investigates the reinforcement learning for the relay selection in the delay-constrained buffer-aided networks. The buffer-aided relay selection significantly improves the outage performance but often at the price of higher latency. On the other hand, modern communication systems such as the Internet of Things often have strict requirement on the latency. It is thus necessary to find relay selection policies to achieve good throughput performance in the buffer-aided relay network while stratifying the delay constraint. With the buffers employed at the relays and delay constraints imposed on the data transmission, obtaining the best relay selection becomes a complicated high-dimensional problem, making it hard for the reinforcement learning to converge. In this paper, we propose the novel decision-assisted deep reinforcement learning to improve the convergence. This is achieved by exploring the a-priori information from the buffer-aided relay system. The proposed approaches can achieve high throughput subject to delay constraints. Extensive simulation results are provided to verify the proposed algorithms

Performance analysis of multi-antenna selection policies using the golden code in multiple-input multiple-output systems

In multiple-input multiple-output (MIMO) systems, multiple-antenna selection has been proposed as a practical scheme for improving the signal transmission quality as well as reducing realisation cost because of minimising the number of radio-frequency chains. In this study, the authors investigate transmit antenna selection for MIMO systems with the Golden Code. Two antenna selection schemes are considered: max-min and max-sum approaches. The outage and pairwise error probability performance of the proposed approaches are analysed. Simulations are also given to verify the analysis. The results show the proposed methods provide useful schemes for antenna selection.</div

Joint Buffer-Aided Hybrid-Duplex Relay Selection and Power Allocation for Secure Cognitive Networks with Double Deep Q-Network

This paper applies the reinforcement learning in the joint relay selection and power allocation in the secure cognitive radio (CR) relay network, where the data buffers and full-duplex jamming are applied at the relay nodes. Two cases are considered: maximizing the throughput with the delay and secrecy constraints, and maximizing the secrecy rate with the delay constraint, respectively. In both cases, the optimization relies on the buffer states, the interference to/from the primary user, and the constraints on the delay and/or secrecy. This makes it mathematically intractable to apply the traditional optimization methods. In this paper, the double deep Q-network (DDQN) is used to solve the above two optimization problems. We also apply the a-priori information in the CR network to improve the DDQN learning convergence. Simulation results show that the proposed scheme outperforms the traditional algorithm significantly

Decode-and-Forward Buffer-Aided Relay Selection in Cognitive Relay Networks

This paper investigates decode-and-forward (DF) buffer-aided relay selection for underlay cognitive relay networks (CRNs) in the presence of both primary transmitter and receiver. We propose a novel buffer-aided relay selection scheme for the CRN, where the best relay is selected with the highest signal-to-interference ratio (SIR) among all available source-to-relay and relay-to-destination links while keeping the interference to the primary destination within a certain level. A new closed-form expression for the outage probability of the proposed relay selection scheme is obtained. Both simulation and theoretical results are shown to confirm performance advantage over the conventional max-min relay selection scheme, making the proposed scheme attractive for CRNs

Buffer-aided relay selection for cooperative NOMA in the internet of things

The nonorthogonal multiple access (NOMA) well improves the spectrum efficiency which is particularly essential in the Internet of Things (IoT) system involving massive number of connections. It has been shown that applying buffers at relays can further increase the throughput in the NOMA relay network. This is however valid only when the channel signal-to-noise ratios (SNRs) are large enough to support the NOMA transmission. While it would be straightforward for the cooperative network to switch between the NOMA and the traditional orthogonal multiple access (OMA) transmission modes based on the channel SNR-s, the best potential throughput would not be achieved. In this paper, we propose a novel prioritization-based buffer-aided relay selection scheme which is able to seamlessly combine the NOMA and OMA transmission in the relay network. The analytical expression of average throughput of the proposed scheme is successfully derived. The proposed scheme significantly improves the data throughput at both low and high SNR ranges, making it an attractive scheme for cooperative NOMA in the IoT

Deep Reinforcement Learning Based Relay Selection in Intelligent Reflecting Surface Assisted Cooperative Networks

This paper proposes a deep reinforcement learning (DRL) based relay selection scheme for cooperative networks with the intelligent reflecting surface (IRS). We consider a practical phase-dependent amplitude model in which the IRS reflection amplitudes vary with the discrete phase-shifts. Furthermore, we apply the relay selection to reduce the signal loss over distance in IRS-assisted networks. To solve the complicated problem of joint relay selection and IRS reflection coefficient optimization, we introduce DRL to learn from the environment to obtain the solution and reduce the computational complexity. Simulation results show that the throughput is significantly improved with the proposed DRL-based algorithm compared to random relay selection and random reflection coefficients methods

Novel deep reinforcement learning-based delay-constrained buffer-aided relay selection in cognitive cooperative networks

In this Letter, a deep reinforcement learning-based approach is proposed for the delay-constrained buffer-aided relay selection in a cooperative cognitive network. The proposed learning algorithm can efficiently solve the complicated relay selection problem, and achieves the optimal throughput when the buffer size and number of relays are large. In particular, the authors use the deep-Q-learning to design an agent to estimate a specific action for each state of the system, which is then utilised to provide an optimum trade-off between throughput and a given delay constraint. Simulation results are provided to demonstrate the advantages of the proposed scheme over conventional selection methods. More specifically, compared to the max-ratio selection criteria, where the relay with the highest signal-to-interference ratio is selected, the proposed scheme achieves a significant throughput gain with higher throughput-delay balance

Max-Ratio Relay Selection in Secure Buffer-Aided Cooperative Wireless Networks

This paper considers the security of transmission in buffer-aided decode-and-forward cooperative wireless networks. An eavesdropper which can intercept the data transmission from both the source and relay nodes is considered to threaten the security of transmission. Finite size data buffers are assumed to be available at every relay in order to avoid having to select concurrently the best source-to-relay and relay-to-destination links. A new max-ratio relay selection policy is proposed to optimize the secrecy transmission by considering all the possible source-to-relay and relay-to-destination links and selecting the relay having the link which maximizes the signal to eavesdropper channel gain ratio. Two cases are considered in terms of knowledge of the eavesdropper channel strengths: exact and average gains, respectively. Closed-form expressions for the secrecy outage probability for both cases are obtained, which are verified by simulations. The proposed max-ratio relay selection scheme is shown to outperform one based on a max-min-ratio relay scheme

Spatial-temporal distribution of deoxynivalenol, aflatoxin B1, and zearalenone in the solid-state fermentation basin of traditional vinegar and their potential correlation with microorganisms

This study revealed the spatial–temporal distribution of deoxynivalenol (DON), aflatoxin B1 (AFB1), and zearalenone (ZEN) during the acetic acid fermentation (AAF) of aromatic vinegar and the corresponding correlation with the microbial community. A total of 324 samples were collected during the AAF process to analyze the mycotoxin content. The average DON content fluctuated during the first 7 d, while the average AFB1 and ZEN levels increased at 5–7 d and 7–11 d, respectively, remaining stable until the end of fermentation. In addition, the significant AFB1 and ZEN content variation was limited to the cross-sectional sampling planes in the fermentation basin, while DON was heterogeneously distributed on the cross-sectional, horizontal, and vertical sampling planes. Furthermore, the redundancy analysis and Spearman correlation coefficients revealed close relationships between three mycotoxins and certain bacterial and fungal species. This study provides new information regarding the mycotoxins during solid-state fermentation of traditional vinegar.</p

Assessment of corrosive attack of Fe9Cr1Mo alloys in pressurised CO2 for prediction of breakaway oxidation

To provide clarity on the poorly-understood mechanism of breakaway oxidation, corrosion of Fe9Cr1Mo steel in pressurised CO2 is quantified and modelled. The temperature range 400–640 ∘C, relevant to nuclear power plants, is emphasised. Attack is in the form of combined oxide scale growth and internal carburisation of the metal. Carbon activity in the metal at its surface exhibits a strong time dependence consistent with the kinetically-limited transport of carbon due to the slow Boudouard reaction. Breakaway is associated with the approach to saturation of the steel with respect to carbon. Diffusion modelling agrees well with steel carbide precipitation observations.</p