Terahertz (THz) communication with ultra-wide available spectrum is a
promising technique that can achieve the stringent requirement of high data
rate in the next-generation wireless networks, yet its severe propagation
attenuation significantly hinders its implementation in practice. Finding beam
directions for a large-scale antenna array to effectively overcome severe
propagation attenuation of THz signals is a pressing need. This paper proposes
a novel approach of federated deep reinforcement learning (FDRL) to swiftly
perform THz-beam search for multiple base stations (BSs) coordinated by an edge
server in a cellular network. All the BSs conduct deep deterministic policy
gradient (DDPG)-based DRL to obtain THz beamforming policy with limited channel
state information (CSI). They update their DDPG models with hidden information
in order to mitigate inter-cell interference. We demonstrate that the cell
network can achieve higher throughput as more THz CSI and hidden neurons of
DDPG are adopted. We also show that FDRL with partial model update is able to
nearly achieve the same performance of FDRL with full model update, which
indicates an effective means to reduce communication load between the edge
server and the BSs by partial model uploading. Moreover, the proposed FDRL
outperforms conventional non-learning-based and existing non-FDRL benchmark
optimization methods