Cycle-accurate evaluation of reconfigurable photonic networks-on-chip

There is little doubt that the most important limiting factors of the performance of next-generation Chip Multiprocessors (CMPs) will be the power efficiency and the available communication speed between cores. Photonic Networks-on-Chip (NoCs) have been suggested as a viable route to relieve the off- and on-chip interconnection bottleneck. Low-loss integrated optical waveguides can transport very high-speed data signals over longer distances as compared to on-chip electrical signaling. In addition, with the development of silicon microrings, photonic switches can be integrated to route signals in a data-transparent way. Although several photonic NoC proposals exist, their use is often limited to the communication of large data messages due to a relatively long set-up time of the photonic channels. In this work, we evaluate a reconfigurable photonic NoC in which the topology is adapted automatically (on a microsecond scale) to the evolving traffic situation by use of silicon microrings. To evaluate this system's performance, the proposed architecture has been implemented in a detailed full-system cycle-accurate simulator which is capable of generating realistic workloads and traffic patterns. In addition, a model was developed to estimate the power consumption of the full interconnection network which was compared with other photonic and electrical NoC solutions. We find that our proposed network architecture significantly lowers the average memory access latency (35% reduction) while only generating a modest increase in power consumption (20%), compared to a conventional concentrated mesh electrical signaling approach. When comparing our solution to high-speed circuit-switched photonic NoCs, long photonic channel set-up times can be tolerated which makes our approach directly applicable to current shared-memory CMPs

On the design of a high-performance adaptive router for CC-NUMA multiprocessors

Copyright © 2003 IEEEThis work presents the design and evaluation of an adaptive packet router aimed at supporting CC-NUMA traffic. We exploit a simple and efficient packet injection mechanism to avoid deadlock, which leads to a fully adaptive routing by employing only three virtual channels. In addition, we selectively use output buffers for implementing the most utilized virtual paths in order to reduce head-of-line blocking. The careful implementation of these features has resulted in a good trade off between network performance and hardware cost. The outcome of this research is a High-Performance Adaptive Router (HPAR), which adequately balances the needs of parallel applications: minimal network latency at low loads and high throughput at heavy loads. The paper includes an evaluation process in which HPAR is compared with other adaptive routers using FIFO input buffering, with or without additional virtual channels to reduce head-of-line blocking. This evaluation contemplates both the VLSI costs of each router and their performance under synthetic and real application workloads. To make the comparison fair, all the routers use the same efficient deadlock avoidance mechanism. In all the experiments, HPAR exhibited the best response among all the routers tested. The throughput gains ranged from 10 percent to 40 percent in respect to its most direct rival, which employs more hardware resources. Other results shown that HPAR achieves up to 83 percent of its theoretical maximum throughput under random traffic and up to 70 percent when running real applications. Moreover, the observed packet latencies were comparable to those exhibited by simpler routers. Therefore, HPAR can be considered as a suitable candidate to implement packet interchange in next generations of CC-NUMA multiprocessors.Valentín Puente, José-Ángel Gregorio, Ramón Beivide, and Cruz Iz

Network unfairness in dragonfly topologies

Dragonfly networks arrange network routers in a two-level hierarchy, providing a competitive cost-performance solution for large systems. Non-minimal adaptive routing (adaptive misrouting) is employed to fully exploit the path diversity and increase the performance under adversarial traffic patterns. Network fairness issues arise in the dragonfly for several combinations of traffic pattern, global misrouting and traffic prioritization policy. Such unfairness prevents a balanced use of the resources across the network nodes and degrades severely the performance of any application running on an affected node. This paper reviews the main causes behind network unfairness in dragonflies, including a new adversarial traffic pattern which can easily occur in actual systems and congests all the global output links of a single router. A solution for the observed unfairness is evaluated using age-based arbitration. Results show that age-based arbitration mitigates fairness issues, especially when using in-transit adaptive routing. However, when using source adaptive routing, the saturation of the new traffic pattern interferes with the mechanisms employed to detect remote congestion, and the problem grows with the network size. This makes source adaptive routing in dragonflies based on remote notifications prone to reduced performance, even when using age-based arbitration.Peer ReviewedPostprint (author's final draft

Modeling and Analysis of the Performance of Exascale Photonic Networks

"This is the peer reviewed version of the following article: Duro, José, Jose A. Pascual, Salvador Petit, Julio Sahuquillo, and María E. Gómez. 2018. Modeling and Analysis of the Performance of Exascale Photonic Networks. Concurrency and Computation: Practice and Experience 31 (21). Wiley. doi:10.1002/cpe.4773, which has been published in final form at https://doi.org/10.1002/cpe.4773. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving."[EN] Photonics technology has become a promising and viable alternative for both on-chip and off-chip interconnection networks of future Exascale systems. Nevertheless, this technology is not mature enough yet in this context, so research efforts focusing on photonic networks are still required to achieve realistic suitable network implementations. In this regard, system-level photonic network simulators can help guide designers to assess the multiple design choices. Most current research is done on electrical network simulators, whose components work widely different from photonics components. In this work, we summarize and compare the working behavior of both technologies which includes the use of optical routers, wavelength-division multiplexing and circuit switching among others. After implementing them into a well-known simulation framework, an extensive simulation study has been carried out using realistic photonic network configurations with synthetic and realistic traffic. Experimental results show that, compared to electrical networks, optical networks can reduce the execution time of the studied real workloads in almost one order of magnitude. Our study also reveals that the photonic configuration highly impacts on the network performance, being the bandwidth per channel and the message length the most important parameters.This work was supported by the ExaNeSt project, funded by the European Union's Horizon 2020 Research and Innovation Program under grant 671553, and by the Spanish Ministerio de Economía y Competitividad (MINECO) and Plan E funds under grant TIN2015-66972-C5-1-R. Pascual was supported by a HiPEAC Collaboration Grant.Duro-Gómez, J.; Pascual Pérez, JA.; Petit Martí, SV.; Sahuquillo Borrás, J.; Gómez Requena, ME. (2019). Modeling and Analysis of the Performance of Exascale Photonic Networks. Concurrency and Computation Practice and Experience. 31(21):1-12. https://doi.org/10.1002/cpe.4773S1123121Top500 website. Accessed January2018.Kodi, A. K., Neel, B., & Brantley, W. C. (2014). Photonic Interconnects for Exascale and Datacenter Architectures. IEEE Micro, 34(5), 18-30. doi:10.1109/mm.2014.62Rumley, S., Nikolova, D., Hendry, R., Li, Q., Calhoun, D., & Bergman, K. (2015). Silicon Photonics for Exascale Systems. Journal of Lightwave Technology, 33(3), 547-562. doi:10.1109/jlt.2014.2363947Shacham, A., Bergman, K., & Carloni, L. P. (2008). Photonic Networks-on-Chip for Future Generations of Chip Multiprocessors. IEEE Transactions on Computers, 57(9), 1246-1260. doi:10.1109/tc.2008.78Batten, C., Joshi, A., Orcutt, J., Khilo, A., Moss, B., Holzwarth, C. W., … Asanovic, K. (2009). Building Many-Core Processor-to-DRAM Networks with Monolithic CMOS Silicon Photonics. IEEE Micro, 29(4), 8-21. doi:10.1109/mm.2009.60WernerS NavaridasJ LujánM.Designing low‐power low‐latency networks‐on‐chip by optimally combining electrical and optical links. Paper presented at: 23rd IEEE International Symposium on High Performance Computer Architecture (HPCA);2016;Austin TX.PucheJ LechagoS PetitS GómezME SahuquilloJ.Accurately modeling a photonic NoC in a detailed CMP simulation framework. Paper presented at: 2016 International Conference on High Performance Computing & Simulation (HPCS);2016;Innsbruck Austria.ChenG ChenH HaurylauM et al.On‐chip copper‐based vs. optical interconnects: delay uncertainty latency power and bandwidth density comparative predictions. Paper presented at: 2006 International Interconnect Technology Conference;2006;Burlingame CA.KatevenisM ChrysosN MarazakisM et al.The ExaNeSt project: Interconnects storage and packaging for exascale systems. Paper presented at: 2016 Euromicro Conference on Digital System Design (DSD);2016;Limassol Cyprus.ConcattoC PascualJA NavaridasJ et al.A CAM-Free Exascalable HPC Router for Low-Energy Communications. Paper presented at: 31st International Conference on Architecture of Computing Systems (ARCS);2018.DuanG‐H FedeliJ‐M KeyvaniniaS ThomsonD et al.10 Gb/s integrated tunable hybrid III‐V/si laser and silicon Mach‐Zehnder modulator. Paper presented at: European Conference and Exhibition on Optical Communications;2012;Amsterdam The Netherlands.DuanGH JanyC Le LiepvreAL et al.Integrated hybrid III‐V/si laser and transmitter. Paper presented at: 2012 International Conference on Indium Phosphide and Related Materials;2012;Santa Barbara CA.Soref, R., & Bennett, B. (1987). Electrooptical effects in silicon. IEEE Journal of Quantum Electronics, 23(1), 123-129. doi:10.1109/jqe.1987.1073206Liu, A., Liao, L., Rubin, D., Nguyen, H., Ciftcioglu, B., Chetrit, Y., … Paniccia, M. (2007). High-speed optical modulation based on carrier depletion in a silicon waveguide. Optics Express, 15(2), 660. doi:10.1364/oe.15.000660Thomson, D. J., Gardes, F. Y., Hu, Y., Mashanovich, G., Fournier, M., Grosse, P., … Reed, G. T. (2011). High contrast 40Gbit/s optical modulation in silicon. Optics Express, 19(12), 11507. doi:10.1364/oe.19.011507Bergman, K., Carloni, L. P., Biberman, A., Chan, J., & Hendry, G. (2014). Photonic Network-on-Chip Design. Integrated Circuits and Systems. doi:10.1007/978-1-4419-9335-9Dong, P., Chen, L., Xie, C., Buhl, L. L., & Chen, Y.-K. (2012). 50-Gb/s silicon quadrature phase-shift keying modulator. Optics Express, 20(19), 21181. doi:10.1364/oe.20.021181DongP LiuX SethumadhavanC et al.224‐Gb/s PDM‐16‐QAM modulator and receiver based on silicon photonic integrated circuits. Paper presented at: Optical Fiber Communication Conference/National Fiber Optic Engineers Conference;2013;Anaheim CA.Navaridas, J., Miguel-Alonso, J., Pascual, J. A., & Ridruejo, F. J. (2011). Simulating and evaluating interconnection networks with INSEE. Simulation Modelling Practice and Theory, 19(1), 494-515. doi:10.1016/j.simpat.2010.08.008Lu, L., Zhao, S., Zhou, L., Li, D., Li, Z., Wang, M., … Chen, J. (2016). 16 × 16 non-blocking silicon optical switch based on electro-optic Mach-Zehnder interferometers. Optics Express, 24(9), 9295. doi:10.1364/oe.24.009295DuroJ PetitS SahuquilloJ GómezME.Modeling a photonic network for exascale computing. Paper presented at: 2017 International Conference on High Performance Computing & Simulation (HPCS);2017;Genoa Italy.Xi, K., Kao, Y.-H., & Chao, H. J. (2012). A Petabit Bufferless Optical Switch for Data Center Networks. Optical Interconnects for Future Data Center Networks, 135-154. doi:10.1007/978-1-4614-4630-9_8KimJ DallyWJ ScottS AbtsD.Technology‐driven highly‐scalable dragonfly topology. Paper presented at: 35th International Symposium on Computer Architecture (ISCA);2008;Beijing China.Essiambre, R.-J., & Tkach, R. W. (2012). Capacity Trends and Limits of Optical Communication Networks. Proceedings of the IEEE, 100(5), 1035-1055. doi:10.1109/jproc.2012.2182970Temprana, E., Myslivets, E., Kuo, B. P.-P., Liu, L., Ataie, V., Alic, N., & Radic, S. (2015). Overcoming Kerr-induced capacity limit in optical fiber transmission. Science, 348(6242), 1445-1448. doi:10.1126/science.aab1781Springel, V. (2005). The cosmological simulation code gadget-2. Monthly Notices of the Royal Astronomical Society, 364(4), 1105-1134. doi:10.1111/j.1365-2966.2005.09655.xPlimpton, S. (1995). Fast Parallel Algorithms for Short-Range Molecular Dynamics. Journal of Computational Physics, 117(1), 1-19. doi:10.1006/jcph.1995.1039Ben‐ItzhakY ZahaviE CidonI KolodnyA.HNOCS: Modular open‐source simulator for heterogeneous NoCs. Paper presented at: 2012 International Conference on Embedded Computer Systems (SAMOS);2012;Samos Greece.HossainH AhmedM Al‐NayeemA IslamTZ AkbarMM.Gpnocsim‐a general purpose simulator for network‐on‐chip. Paper presented at: 2007 International Conference on Information and Communication Technology;2007;Dhaka Bangladesh.JainL Al‐HashimiB GaurMS LaxmiV NarayananA.NIRGAM: A simulator for NoC interconnect routing and application modeling. Paper presented at: Design Automation and Test in Europe Conference;2007;Nice France.ChanJ HendryG BibermanA BergmanK CarloniLP.PhoenixSim: A simulator for physical‐layer analysis of chip‐scale photonic interconnection networks. In: Proceedings of the Conference on Design Automation and Test in Europe;2010;Dresden Germany.RumleyS BahadoriM WenK NikolovaD BergmanK.PhoenixSim: crosslayer design and modeling of silicon photonic interconnects. In: Proceedings of the 1st International Workshop on Advanced Interconnect Solutions and Technologies for Emerging Computing Systems;2016;Prague Czech Republic.VargaA HornigR.An overview of the OMNeT++ simulation environment. In: Proceedings of the 1st International Conference on Simulation Tools and Techniques for Communications Networks and Systems & Workshops;2008;Marseille France.SunC ChenCHO KurianG et al.DSENT‐a tool connecting emerging photonics with electronics for opto‐electronic networks‐on‐chip modeling. Paper presented at: 2012 IEEE/ACM Sixth International Symposium on Networks‐on‐Chip;2012;Copenhagen Denmark.Ma, X., Yu, J., Hua, X., Wei, C., Huang, Y., Yang, L., … Yang, J. (2014). LioeSim: A Network Simulator for Hybrid Opto-Electronic Networks-on-Chip Analysis. Journal of Lightwave Technology, 32(22), 4301-4310. doi:10.1109/jlt.2014.2356515KahngAB LiB PehL‐S SamadiK.ORION 2.0: a fast and accurate NoC power and area model for early‐stage design space exploration. In: Proceedings of the Conference on Design Automation and Test in Europe;2009;Nice France.Chan, J., Hendry, G., Bergman, K., & Carloni, L. P. (2011). Physical-Layer Modeling and System-Level Design of Chip-Scale Photonic Interconnection Networks. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 30(10), 1507-1520. doi:10.1109/tcad.2011.215715

An Energy and Performance Exploration of Network-on-Chip Architectures

In this paper, we explore the designs of a circuit-switched router, a wormhole router, a quality-of-service (QoS) supporting virtual channel router and a speculative virtual channel router and accurately evaluate the energy-performance tradeoffs they offer. Power results from the designs placed and routed in a 90-nm CMOS process show that all the architectures dissipate significant idle state power. The additional energy required to route a packet through the router is then shown to be dominated by the data path. This leads to the key result that, if this trend continues, the use of more elaborate control can be justified and will not be immediately limited by the energy budget. A performance analysis also shows that dynamic resource allocation leads to the lowest network latencies, while static allocation may be used to meet QoS goals. Combining the power and performance figures then allows an energy-latency product to be calculated to judge the efficiency of each of the networks. The speculative virtual channel router was shown to have a very similar efficiency to the wormhole router, while providing a better performance, supporting its use for general purpose designs. Finally, area metrics are also presented to allow a comparison of implementation costs