Overcoming the Challenges for Multichip Integration: A Wireless Interconnect Approach

The physical limitations in the area, power density, and yield restrict the scalability of the single-chip multicore system to a relatively small number of cores. Instead of having a large chip, aggregating multiple smaller chips can overcome these physical limitations. Combining multiple dies can be done either by stacking vertically or by placing side-by-side on the same substrate within a single package. However, in order to be widely accepted, both multichip integration techniques need to overcome significant challenges. In the horizontally integrated multichip system, traditional inter-chip I/O does not scale well with technology scaling due to limitations of the pitch. Moreover, to transfer data between cores or memory components from one chip to another, state-of-the-art inter-chip communication over wireline channels require data signals to travel from internal nets to the peripheral I/O ports and then get routed over the inter-chip channels to the I/O port of the destination chip. Following this, the data is finally routed from the I/O to internal nets of the target chip over a wireline interconnect fabric. This multi-hop communication increases energy consumption while decreasing data bandwidth in a multichip system. On the other hand, in vertically integrated multichip system, the high power density resulting from the placement of computational components on top of each other aggravates the thermal issues of the chip leading to degraded performance and reduced reliability. Liquid cooling through microfluidic channels can provide cooling capabilities required for effective management of chip temperatures in vertical integration. However, to reduce the mechanical stresses and at the same time, to ensure temperature uniformity and adequate cooling competencies, the height and width of the microchannels need to be increased. This limits the area available to route Through-Silicon-Vias (TSVs) across the cooling layers and make the co-existence and co-design of TSVs and microchannels extreamly challenging. Research in recent years has demonstrated that on-chip and off-chip wireless interconnects are capable of establishing radio communications within as well as between multiple chips. The primary goal of this dissertation is to propose design principals targeting both horizontally and vertically integrated multichip system to provide high bandwidth, low latency, and energy efficient data communication by utilizing mm-wave wireless interconnects. The proposed solution has two parts: the first part proposes design methodology of a seamless hybrid wired and wireless interconnection network for the horizontally integrated multichip system to enable direct chip-to-chip communication between internal cores. Whereas the second part proposes a Wireless Network-on-Chip (WiNoC) architecture for the vertically integrated multichip system to realize data communication across interlayer microfluidic coolers eliminating the need to place and route signal TSVs through the cooling layers. The integration of wireless interconnect will significantly reduce the complexity of the co-design of TSV based interconnects and microchannel based interlayer cooling. Finally, this dissertation presents a combined trade-off evaluation of such wireless integration system in both horizontal and vertical sense and provides future directions for the design of the multichip system

Improving processor efficiency through thermal modeling and runtime management of hybrid cooling strategies

One of the main challenges in building future high performance systems is the ability to maintain safe on-chip temperatures in presence of high power densities. Handling such high power densities necessitates novel cooling solutions that are significantly more efficient than their existing counterparts. A number of advanced cooling methods have been proposed to address the temperature problem in processors. However, tradeoffs exist between performance, cost, and efficiency of those cooling methods, and these tradeoffs depend on the target system properties. Hence, a single cooling solution satisfying optimum conditions for any arbitrary system does not exist. This thesis claims that in order to reach exascale computing, a dramatic improvement in energy efficiency is needed, and achieving this improvement requires a temperature-centric co-design of the cooling and computing subsystems. Such co-design requires detailed system-level thermal modeling, design-time optimization, and runtime management techniques that are aware of the underlying processor architecture and application requirements. To this end, this thesis first proposes compact thermal modeling methods to characterize the complex thermal behavior of cutting-edge cooling solutions, mainly Phase Change Material (PCM)-based cooling, liquid cooling, and thermoelectric cooling (TEC), as well as hybrid designs involving a combination of these. The proposed models are modular and they enable fast and accurate exploration of a large design space. Comparisons against multi-physics simulations and measurements on testbeds validate the accuracy of our models (resulting in less than 1C error on average) and demonstrate significant reductions in simulation time (up to four orders of magnitude shorter simulation times). This thesis then introduces temperature-aware optimization techniques to maximize energy efficiency of a given system as a whole (including computing and cooling energy). The proposed optimization techniques approach the temperature problem from various angles, tackling major sources of inefficiency. One important angle is to understand the application power and performance characteristics and to design management techniques to match them. For workloads that require short bursts of intense parallel computation, we propose using PCM-based cooling in cooperation with a novel Adaptive Sprinting technique. By tracking the PCM state and incorporating this information during runtime decisions, Adaptive Sprinting utilizes the PCM heat storage capability more efficiently, achieving 29\% performance improvement compared to existing sprinting policies. In addition to the application characteristics, high heterogeneity in on-chip heat distribution is an important factor affecting efficiency. Hot spots occur on different locations of the chip with varying intensities; thus, designing a uniform cooling solution to handle worst-case hot spots significantly reduces the cooling efficiency. The hybrid cooling techniques proposed as part of this thesis address this issue by combining the strengths of different cooling methods and localizing the cooling effort over hot spots. Specifically, the thesis introduces LoCool, a cooling system optimizer that minimizes cooling power under temperature constraints for hybrid-cooled systems using TECs and liquid cooling. Finally, the scope of this work is not limited to existing advanced cooling solutions, but it also extends to emerging technologies and their potential benefits and tradeoffs. One such technology is integrated flow cell array, where fuel cells are pumped through microchannels, providing both cooling and on-chip power generation. This thesis explores a broad range of design parameters including maximum chip temperature, leakage power, and generated power for flow cell arrays in order to maximize the benefits of integrating this technology with computing systems. Through thermal modeling and runtime management techniques, and by exploring the design space of emerging cooling solutions, this thesis provides significant improvements in processor energy efficiency.2018-07-09T00:00:00

Can Cooling Technology Save Many-Core Parallel Programming from Its Programming Woes?

An abstract of this work will be presented at the Compiler, Architecture and Tools Conference (CATC), Intel Development Center, Haifa, Israel November 23, 2015.This paper is advancing the following premise (henceforth, "vision"): that it is feasible to greatly enhance data movement in the short term, and do it in ways that would be both power efficient and pragmatic in the long term. The paper spells this premise out in greater detail: 1. it is feasible to build first generations of a variety of (power-inefficient) designs for which data movement will not be a restriction and begin application software development for them; 2. growing reliance on silicon compatible photonic technologies, and feasible advances in them with proper investment, will allow reduction of power consumption in these design by several orders of magnitude; 3. successful high performance application software, the ease of programming demonstrated and growing adoption by customers, software vendors and programmers will incentivize (hardware vendor) investment in new application-software-compatible generations of these designs (a new "software spiral" a la former Intel CEO, Andy Grove) with further reduction of power consumption in each generation; 4. microfluidic cooling is instrumental for enabling item 1, as well as for midwifing this overall vision. The opening paragraph of the paper provides a preamble to that vision, the body of the paper supports it and the paragraph "Moore's-Law-type vision" summarizes it. The scope of the paper is a bit forward looking and it may not exactly fit any particular community. However, its new directions for interaction among architecture and programming may suggest new horizons for representing and exposing a greater variety of data and task parallelism.National Science Foundatio

GreenCool: An Energy-Efficient Liquid Cooling Design Technique for 3-D MPSoCs Via Channel Width Modulation

Liquid cooling using interlayer microchannels has appeared as a viable and scalable packaging technology for 3-D multiprocessor system-on-chips (MPSoCs). Microchannel-based liquid cooling, however, can substantially increase the on-chip thermal gradients, which are undesirable for reliability, performance, and cooling efficiency. In this paper, we present GreenCool, an optimal design methodology for liquid-cooled 3-D MPSoCs. GreenCool simultaneously minimizes the cooling energy for a given system while maintaining thermal gradients and peak temperatures under safe limits. This is accomplished by tuning the heat transfer characteristics of the microchannels using channel width modulation. Channel width modulation is compatible with the current process technologies and incurs minimal additional fabrication costs. Through an extensive set of experiments, we show that channel width modulation is capable of complementing and enhancing the benefits of temperature-aware floorplanning. We also experiment with a 16-core 3-D system with stacked dynamic random-access memory, for which GreenCool improves energy efficiency by up to 53% with respect to no channel modulation

Feasibility Study of Scaling an XMT Many-Core

The reason for recent focus on communication avoidance is that high rates of data movement become infeasible due to excessive power dissipation. However, shifting the responsibility of minimizing data movement to the parallel algorithm designer comes at significant costs to programmer’s productivity, as well as: (i) reduced speedups and (ii) the risk of repelling application developers from adopting parallelism. The UMD Explicit Multi-Threading (XMT) framework has demonstrated advantages on ease of parallel programming through its support of PRAM-like programming, combined with strong, often unprecedented speedups. Such programming and speedups involve considerable data movement between processors and shared memory. Another reason that XMT is a good test case for a study of data movement is that XMT permits isolation and direct study of most of its data movement (and its power dissipation). Our new results demonstrate that an XMT single-chip many-core processor with tens of thousands of cores and a high throughput network on chip is thermally feasible, though at some cost. This leads to a perhaps game-changing outcome: instead of imposing upfront strict restrictions on data movement, as advocated in a recent report from the National Academies, opt for due diligence that accounts for the full impact on cost. For example, does the increased cost due to communication avoidance (including programmer’s productivity, reduced speedups and desertion risk) indeed offset the cost of the solution we present? More specifically, we investigate in this paper the design of an XMT many-core for 3D VLSI with microfluidic cooling. We used state-of-the-art simulation tools to model the power and thermal properties of such an architecture with 8k to 64k lightweight cores, requiring between 2 and 8 silicon layers. Inter-chip communication using silicon compatible photonics is also considered. We found that, with the use of microfluidic cooling, power dissipation becomes a cost issue rather than a feasibility constraint. Robustness of the results is also discussed.DARPA, NSF, NI

Transport Characteristics of Pin Fin Enhanced Microgaps under Single and Two Phase Cooling

Microfluidic convection cooling is a promising technique for future high power microprocessors, radio-frequency (RF) transceivers, solid-state lasers, and light emitting diodes (LED). Three-dimensional (3D) stacking of chips is a configuration that allows many performance benefits. A microgap with circulating fluid is a promising cooling arrangement that can be incorporated within a 3D chip stack. Although studies have examined the thermal characteristics of microgaps under both single-phase and two-phase convection, the characteristics and benefits of microgaps with surface enhancement features have not been fully explored. In this work, firstly, the single phase thermal/fluid characteristics of microgaps with staggered pin fin arrays are studied. The effects of the pin fin dimensions including diameter, transversal and longitudinal spacing, and height are investigated computationally and experimentally over a range of Reynolds number (Re) 22-357. Micropin fin arrays investigated have pin diameter of 100 μm, pitch/ diameter ratios of 1.5 ~ 2.25, and height/ diameter ratios of 1.5 ~ 2.25. Correlations of friction factor (f) and Colburn j factor for these dense arrays of micro pins have been developed. Subsequently, microfluidic cooling with staggered pin fin arrays is employed in functional 3D integrated circuit (ICs). Thermal and electrical performance of a CMOS chip in terms of temperature and leakage power under realistic operating conditions are studied. Both experimental and modeling results show that microfluidic cooling could significantly decrease the chip temperature and leakage power, thus increasing the chip performance. Lastly, two-phase cooling is studied with dielectric fluid HFE-7200 as a baseline with mass flux from 354.5 kg/m2-s to 576.3 kg/m2-s. Critical heat flux (CHF) increases with increasing mass flux but decreases with decreasing gap height. Nonuniform heating will cause nonuniform flow with a decrease of mass flux in high power area, which decreases the thermal performance. The effects of fluid mixture (HFE-7200/Methanol) on thermal performance are studied with mass fraction of Methanol from 8.5% to 35.8%. A very small amount of addition of Methanol (8.5% mass fraction) can significantly increase the thermal performance due to the sharp decrease of saturation temperature and increase of effective thermal conductivity and latent heat. However, the Marangoni effect caused by the concentration gradient deteriorates the CHF.Ph.D

Integrated Microfluidic Power Generation and Cooling for Bright Silicon MPSoCs

The soaring demand for computing power in our digital information age has produced as collateral undesirable effect a surge in power consumption and heat density for computing servers. Accordingly, 30-40% of the energy consumed in state-of-the-art servers is dissipated in cooling. The remaining energy is used for computation, and causes the temperature ramp-up to operating conditions that already preclude operating all the cores at maximum performance levels, in order to prevent system overheating and failures. This situation is set to worsen as shipments of high-end (i.e., even denser) many-core servers are increasing at a 25% compound annual growth rate. Thus, state-of-the-art worst-case power and cooling delivery solutions on servers are reaching their limits and it will no longer be possible to power up simultaneously all the available on-chip cores (situation known as the existence of "dark silicon"); hence, drastically limiting the benefits of technology scaling. This presentation aims to completely revise the prevailing worst-case power and cooling provisioning paradigm for servers by championing a disruptive approach to computing server architecture design that prevents dark silicon. This proposed approach integrates a flexible heterogeneous many-core architecture template with an on-chip microfluidic fuel cell network for joint cooling delivery and power supply (i.e., local power generation and delivery), as well as a holistic power-temperature model predictive controller exploiting the server software stack, in order to achieve scalable and energy-minimal server architectures. Thanks to the disruptive system-level many-core architecture with microfluidic power and cooling delivery, as well as the complementary temperature control, we can envision the removal of the current limits of power delivery and heat dissipation in server designs, subsequently avoiding dark silicon in future servers and enabling new perspectives in future energy-proportional server designs

Architectural-Physical Co-Design of 3D CPUs with Micro-Fluidic Cooling

The performance, energy efficiency and cost improvements due to traditional technology scaling have begun to slow down and present diminishing returns. Underlying reasons for this trend include fundamental physical limits of transistor scaling, the growing significance of quantum effects as transistors shrink, and a growing mismatch between transistors and interconnects regarding size, speed and power. Continued Moore's Law scaling will not come from technology scaling alone, and must involve improvements to design tools and development of new disruptive technologies such as 3D integration. 3D integration presents potential improvements to interconnect power and delay by translating the routing problem into a third dimension, and facilitates transistor density scaling independent of technology node. Furthermore, 3D IC technology opens up a new architectural design space of heterogeneously-integrated high-bandwidth CPUs. Vertical integration promises to provide the CPU architectures of the future by integrating high performance processors with on-chip high-bandwidth memory systems and highly connected network-on-chip structures. Such techniques can overcome the well-known CPU performance bottlenecks referred to as memory and communication wall. However the promising improvements to performance and energy efficiency offered by 3D CPUs does not come without cost, both in the financial investments to develop the technology, and the increased complexity of design. Two main limitations to 3D IC technology have been heat removal and TSV reliability. Transistor stacking creates increases in power density, current density and thermal resistance in air cooled packages. Furthermore the technology introduces vertical through silicon vias (TSVs) that create new points of failure in the chip and require development of new BEOL technologies. Although these issues can be controlled to some extent using thermal-reliability aware physical and architectural 3D design techniques, high performance embedded cooling schemes, such as micro-fluidic (MF) cooling, are fundamentally necessary to unlock the true potential of 3D ICs. A new paradigm is being put forth which integrates the computational, electrical, physical, thermal and reliability views of a system. The unification of these diverse aspects of integrated circuits is called Co-Design. Independent design and optimization of each aspect leads to sub-optimal designs due to a lack of understanding of cross-domain interactions and their impacts on the feasibility region of the architectural design space. Co-Design enables optimization across layers with a multi-domain view and thus unlocks new high-performance and energy efficient configurations. Although the co-design paradigm is becoming increasingly necessary in all fields of IC design, it is even more critical in 3D ICs where, as we show, the inter-layer coupling and higher degree of connectivity between components exacerbates the interdependence between architectural parameters, physical design parameters and the multitude of metrics of interest to the designer (i.e. power, performance, temperature and reliability). In this dissertation we present a framework for multi-domain co-simulation and co-optimization of 3D CPU architectures with both air and MF cooling solutions. Finally we propose an approach for design space exploration and modeling within the new Co-Design paradigm, and discuss the possible avenues for improvement of this work in the future

Hybrid microfluidic cooling and thermal isolation technologies for 3D ICs

A key challenge for three dimensional (3D) integrated circuits (ICs) is thermal management. There are two main thermal challenges in typical 3D ICs. First, in the homogeneous integration with multiple high-power tiers, an effective cooling solution that scales with the number of dice in the stack is needed. Second, in the heterogeneous integration, an effective thermal isolation solution is needed to ‘protect’ the low-power tier from the high-power tier. This research focuses to address these two thermal challenges through hybrid microfluidic cooling and thermal isolation technologies. Within-tier microfluidic cooling is proposed and demonstrated to cool a stack with multiple high-power tiers. Electrical thermal co-analysis is performed to understand the trade-offs between through silicon via (TSV) parasitics and heat sink performance. A TSV-compatible micropin-fin heat sink is designed, fabricated and thermally characterized in a single tier, and benchmarked with a conventional air-cooled heat sink. The designed heat sink has a thermal resistance of 0.269 K·cm2/W at a flow rate of 70 mL/min. High aspect ratios TSVs (18:1) are integrated in the micropin-fins. Within-tier microfluidic cooling is then implemented in 3D stacks to emulate different heating scenarios, such as memory-on-processor and processor-on-processor. Air gap and mechanically flexible interconnects (MFIs) are proposed for the first time to decrease the vertical thermal coupling between high-power (e.g. processor) and low-power tiers (e.g. memory or nanophotonics). A two-tier testbed with the proposed thermal isolation technology is designed, fabricated and tested. Compared with conventional 3D integration approach, thermal isolation technology helps reduce the temperature at a fixed location in the low-tier by 12.9 °C. The resistance of a single MFI is measured to be 46.49 mΩ.Ph.D

Temperature-Aware Design and Management for 3D Multi-Core Architectures

Vertically-integrated 3D multiprocessors systems-on-chip (3D MPSoCs) provide the means to continue integrating more functionality within a unit area while enhancing manufacturing yields and runtime performance. However, 3D MPSoCs incur amplified thermal challenges that undermine the corresponding reliability. To address these issues, several advanced cooling technologies, alongside temperature-aware design-time optimizations and run-time management schemes have been proposed. In this monograph, we provide an overall survey on the recent advances in temperature-aware 3D MPSoC considerations. We explore the recent advanced cooling strategies, thermal modeling frameworks, design-time optimizations and run-time thermal management schemes that are primarily targeted for 3D MPSoCs. Our aim of proposing this survey is to provide a global perspective, highlighting the advancements and drawbacks on the recent state-of-the-ar