A HEURISTIC FOR OPTIMIZING THE PHYSICAL LAYOUT AND NETWORK TOPOLOGY OF INTEGRATED 3D MULTI-CHIP SYSTEMS UNDER TEMPERATURE CONSTRAINTS

M.S. Thesis. University of Hawaiʻi at Mānoa 2018

Overcoming the Challenges for Multichip Integration: A Wireless Interconnect Approach

The physical limitations in the area, power density, and yield restrict the scalability of the single-chip multicore system to a relatively small number of cores. Instead of having a large chip, aggregating multiple smaller chips can overcome these physical limitations. Combining multiple dies can be done either by stacking vertically or by placing side-by-side on the same substrate within a single package. However, in order to be widely accepted, both multichip integration techniques need to overcome significant challenges. In the horizontally integrated multichip system, traditional inter-chip I/O does not scale well with technology scaling due to limitations of the pitch. Moreover, to transfer data between cores or memory components from one chip to another, state-of-the-art inter-chip communication over wireline channels require data signals to travel from internal nets to the peripheral I/O ports and then get routed over the inter-chip channels to the I/O port of the destination chip. Following this, the data is finally routed from the I/O to internal nets of the target chip over a wireline interconnect fabric. This multi-hop communication increases energy consumption while decreasing data bandwidth in a multichip system. On the other hand, in vertically integrated multichip system, the high power density resulting from the placement of computational components on top of each other aggravates the thermal issues of the chip leading to degraded performance and reduced reliability. Liquid cooling through microfluidic channels can provide cooling capabilities required for effective management of chip temperatures in vertical integration. However, to reduce the mechanical stresses and at the same time, to ensure temperature uniformity and adequate cooling competencies, the height and width of the microchannels need to be increased. This limits the area available to route Through-Silicon-Vias (TSVs) across the cooling layers and make the co-existence and co-design of TSVs and microchannels extreamly challenging. Research in recent years has demonstrated that on-chip and off-chip wireless interconnects are capable of establishing radio communications within as well as between multiple chips. The primary goal of this dissertation is to propose design principals targeting both horizontally and vertically integrated multichip system to provide high bandwidth, low latency, and energy efficient data communication by utilizing mm-wave wireless interconnects. The proposed solution has two parts: the first part proposes design methodology of a seamless hybrid wired and wireless interconnection network for the horizontally integrated multichip system to enable direct chip-to-chip communication between internal cores. Whereas the second part proposes a Wireless Network-on-Chip (WiNoC) architecture for the vertically integrated multichip system to realize data communication across interlayer microfluidic coolers eliminating the need to place and route signal TSVs through the cooling layers. The integration of wireless interconnect will significantly reduce the complexity of the co-design of TSV based interconnects and microchannel based interlayer cooling. Finally, this dissertation presents a combined trade-off evaluation of such wireless integration system in both horizontal and vertical sense and provides future directions for the design of the multichip system

Development of Silicon Photonic Multi Chip Module Transceivers

The exponential growth of data generation–driven in part by the proliferation of applications such as high definition streaming, artificial intelligence, and the internet of things–presents an impending bottleneck for electrical interconnects to fulfill data center bandwidth demands. Links now require bandwidths in excess of multiple Tbps while operating on the order of picojoules per bit, in addition to constraints on areal bandwidth densities and pin I/O bandwidth densities. Optical communications built on a silicon photonic platform offers a potential solution to develop power efficient, high bandwidth, low attenuation, small footprint links, all while building off the mature CMOS ecosystem. The development of silicon photonic foundries supporting multi project wafer runs with associated process design kit components supports a path towards widespread commercial production by increasing production volume while reducing fabrication and development costs. While silicon photonics can always be improved in terms of performance and yield, one of the central challenges is the integration of the silicon photonic integrated circuits with the driving electronic integrated circuits and data generating compute nodes such as CPUs, FPGAs, and ASICs. The co-packaging of the photonics with the electronics is crucial for adoption of silicon photonics in datacenters, as improper integration negates all the potential benefits of silicon photonics. The work in this dissertation is centered around the development of silicon photonic multi chip module transceivers to aid in the deployment of silicon photonics within data centers. Section one focuses on silicon photonic integration and highlights multiple integrated transceiver prototypes. The central prototype features a photonic integrated circuit with bus waveguides with WDM microdisk modulators for the transmitter and WDM demuxes with drop ports to photodiodes for the receiver. The 2.5D integrated prototype utilizes a thinned silicon interposer and TIA electronic integrated circuits. The architecture, integration, characterization, performance, and scalability of the prototype are discussed. The development of this first prototype identified key design considerations necessary for designing multi chip module silicon photonic prototypes, which will be addressed in this section. Finally, other multi chip module silicon photonic prototypes will be overviewed. These include a 2.5D integrated transceiver with a different electronic integrated circuit TIA, a 3D integrated receiver, an active interposer network on chip, and a 2.5D integrated transceiver with custom electronic integrated circuits. Section two focuses on research that supports the development of silicon photonic transceivers. The thermal crosstalk from neighboring microdisk modulators as a function of modulator pitch is investigated. As modulators are placed at denser pitches to accommodate areal bandwidth density requirements in transceivers, this thermal crosstalk will become significant. In this section, designs and results from several iterations of custom microring modulators are reported. Custom microring modulators allow for scaling up the number of channels in microring transceivers by offering the ability to fabricate variable resonances and provide a platform for further innovation in bandwidth, free spectral range, and energy efficiency. The designs and results of higher order modulation format modulators, both microring based and Mach Zehnder based, are discussed. High order modulators offer a path towards scaling transceiver total throughput without having to increase the channel counts or component bandwidth. Together, the work in these two sections supports the development of silicon photonic transceivers to aid in the adoption of silicon photonics into data generating systems

Photonic Interconnection Networks for Applications in Heterogeneous Utility Computing Systems

Growing demands in heterogeneous utility computing systems in future cloud and high performance computing systems are driving the development of processor-hardware accelerator interconnects with greater performance, flexibility, and dynamism. Recent innovations in the field of utility computing have led to an emergence in the use of heterogeneous compute elements. By leveraging the computing advantages of hardware accelerators alongside typical general purpose processors, performance efficiency can be maximized. The network linking these compute nodes is increasingly becoming the bottleneck in these architectures, limiting the hardware accelerators to be restricted to localized computing. A high-bandwidth, agile interconnect is an imperative enabler for hardware accelerator delocalization in heterogeneous utility computing. A redesign of these systems' interconnect and architecture will be essential to establishing high-bandwidth, low-latency, efficient, and dynamic heterogeneous systems that can meet the challenges of next-generation utility computing. By leveraging an optics-based approach, this dissertation presents the design and implementation of optically-connected hardware accelerators (OCHA) that exploit the distance-independent energy dissipation and bandwidth density of photonic transceivers, in combination with the flexibility, efficiency and data parallelization offered by optical networks. By replacing the electronic buses with an optical interconnection network, architectures that delocalize hardware accelerators can be created that are otherwise infeasible. With delocalized optically-connected hardware accelerator nodes accessible by processors at run time, the system can alleviate the network latency issues plague current heterogeneous systems. Accelerators that would otherwise sit idle, waiting for it's master CPU to feed it data, can instead operate at high utilization rates, leading to dramatic improvements in overall system performance. This work presents a prototype optically-connect hardware accelerator module and custom optical-network-aware, dynamic hardware accelerator allocator that communicate transparently and optically across an optical interconnection network. The hardware accelerators and processor are optimized to enable hardware acceleration across an optical network using fast packet-switching. The versatility of the optical network enables additional performance benefits including optical multicasting to exploit the data parallelism found in many accelerated data sets. The integration of hardware acceleration, heterogeneous computing, and optics constitutes a critical step for both computing and optics. The massive data parallelism, application dependent-location and function, as well as network latency, and bandwidth limitations facing networks today complement well with the strength of optical communications-based systems. Moreover, ongoing efforts focusing on development of low-cost optical components and subsystems that are suitable for computing environment may benefit from the high-volume heterogeneous computing market. This work, therefore, takes the first steps in merging the areas of hardware acceleration and optics by developing architectures, protocols, and systems to interface with the two technologies and demonstrating areas of potential benefits and areas for future work. Next-generation heterogeneous utility computing systems will indubitably benefit from the use of efficient, flexible and high-performance optically connect hardware acceleration

Evaluating Techniques for Wireless Interconnected 3D Processor Arrays

In this thesis the viability of a wireless interconnect network for a highly parallel computer is investigated. The main theme of this thesis is to project the performance of a wireless network used to connect the processors in a parallel machine of such design. This thesis is going to investigate new design opportunities a wireless interconnect network can offer for parallel computing. A simulation environment is designed and implemented to carry out the tests. The results have shown that if the available radio spectrum is shared effectively between building blocks of the parallel machine, there are substantial chances to achieve high processor utilisation. The results show that some factors play a major role in the performance of such a machine. The size of the machine, the size of the problem and the communication and computation capabilities of each element of the machine are among those factors. The results show these factors set a limit on the number of nodes engaged in some classes of tasks. They have shown promising potential for further expansion and evolution of our idea to new architectural opportunities, which is discussed by the end of this thesis. To build a real machine of this type the architects would need to solve a number of challenging problems including heat dissipation, delivering electric power and Chip/board design; however, these issues are not part of this thesis and will be tackled in future

Process Variation in Silicon Photonic Devices

The high index contrast of the silicon - silicon dioxide material system allows for dense integration of optical waveguide devices. Possible applications include intra-chip, inter-chip and fiber optic interconnection systems. Optical intra-chip interconnections become more desirable as the complementary metal-oxide-semiconductor (CMOS) circuit density puts ever tighter constraint on on-chip interconnection performance. Board level, rack level and rack-to-rack data center interconnections are ever more constrained by space and bandwidth to which silicon photonic modules may offer an improvement. As fiber optic systems serve smaller and smaller area systems, integrated switching systems that are enabled by silicon photonic devices involving wavelength division multiplexing (WDM) become more desirable. In this thesis, we firstly take a brief review of the development history of information technology, optical communication and silicon photonics. Secondly we examine the optical performance of an array of photonic devices which are the basic building blocks for silicon photonic circuits. Thirdly we turn the attention to the fabrication related issues. Silicon photonic circuits are prone to the thermal and fabrication induced process variations. We discover the process variation exhibits a “random walk” pattern with spatial extent at wafer scale. Fourthly we propose a simple method to extract fundamental parameters out of fabricated silicon photonic devices. Based on the systemic wafer-scale measurement results, our method combines the advantage of both numerical simulation and simple analytical modeling techniques. Lastly, we propose a variation-aware on-chip interconnect design for multi-core processors. This design adapts to on-chip thermal and process variation effects, pointing to the improvement of wafer-scale fabrication yield and interconnect network communication throughput