Densely deployed small cell networks will address the growing demand for broadband mobile connectivity, by increasing access network capacity and coverage. However, most potential small cell base station (SCBS) locations do not have existing telecommunication infrastructure. Providing backhaul connectivity to core networks is therefore a challenge. Millimeter wave (mmW) technologies operated at 30-90GHz are currently being considered to provide low-cost, flexible, high-capacity and reliable backhaul solutions using existing roof-mounted backhaul aggregation sites. Using intelligent mmW radio devices and massive multiple-input multiple-output (MIMO), for enabling point-to-multipoint (PtMP) operation, is considered in this research. The core aim of this research is to develop an architecture of an intelligent non-line-sight (NLOS) small cell backhaul (SCB) system based on mmW and massive MIMO technologies, and supporting intelligent algorithms to facilitate reliable NLOS street-to-rooftop NLOS SCB connectivity. In the proposed architecture, diffraction points are used as signal anchor points between backhaul radio devices. In the new architecture the integration of these technologies is considered. This involves the design of efficient artificial intelligence algorithms to enable backhaul radio devices to autonomously select suitable NLOS propagation paths, find an optimal number of links that meet the backhaul performance requirements and determine an optimal number of diffractions points capable of covering predetermined SCB locations. Throughout the thesis, a number of algorithms are developed and simulated using the MATLAB application. This thesis mainly investigates three key issues: First, a novel intelligent NLOS SCB architecture, termed the cognitive NLOS SCB (CNSCB) system is proposed to enable street-to-rooftop NLOS connectivity using predetermined diffraction points located on roof edges. Second, an algorithm to enable the autonomous creation of multiple-paths, evaluate the performance of each link and determine an optimal number of possible paths per backhaul link is developed. Third, an algorithm to determine the optimal number of diffraction points that can cover an identified SCBS location is also developed. Also, another investigated issue related to the operation of the proposed architecture is its energy efficiency, and its performance is compared to that of a point-to-point (PtP) architecture. The proposed solutions were examined using analytical models, simulations and experimental work to determine the strength of the street-to-rooftop backhaul links and their ability to meet current and future SCB requirements. The results obtained showed that reliable multiple NLOS links can be achieved using device intelligence to guide radio signals along the propagation path. Furthermore, the PtMP architecture is found to be more energy efficient than the PtP architecture. The proposed architecture and algorithms offer a novel backhaul solution for outdoor urban small cells. Finally, this research shows that traditional techniques of addressing the demand for connectivity, which consisted of improving or evolving existing solutions, may nolonger be applicable in emerging communication technologies. There is therefore need to consider new ways of solving the emerging challenges
