Millimeter wave (mmWave) communications provide great potential for
next-generation cellular networks to meet the demands of fast-growing mobile
data traffic with plentiful spectrum available. However, in a mmWave cellular
system, the shadowing and blockage effects lead to the intermittent
connectivity, and the handovers are more frequent. This paper investigates an
``all-mmWave'' cloud radio access network (cloud-RAN), in which both the
fronthaul and the radio access links operate at mmWave. To address the
intermittent transmissions, we allow the mobile users (MUs) to establish
multiple connections to the central unit over the remote radio heads (RRHs).
Specifically, we propose a multipath transmission framework by leveraging the
``all-mmWave'' cloud-RAN architecture, which makes decisions of the RRH
association and the packet transmission scheduling according to the
time-varying network statistics, such that a MU experiences the minimum
queueing delay and packet drops. The joint RRH association and transmission
scheduling problem is formulated as a Markov decision process (MDP). Due to the
problem size, a low-complexity online learning scheme is put forward, which
requires no a priori statistic information of network dynamics. Simulations
show that our proposed scheme outperforms the state-of-art baselines, in terms
of average queue length and average packet dropping rate