The Convergence of Ethernet and Ethernot: A 10-Gigabit Ethernet Perspective ∗

Abstract

Recently, a vast number of interconnect technologies such as InfiniBand, Myrinet and Quadrics have been introduced into the system-area network (SAN) environment; the primary driving requirements of this environment being high-performance and a feature-rich interface. Ethernet, on the other hand, is already the ubiquitous technology for wide-area network (WAN) environments. Traditionally, SAN technologies had been shut off from the WAN environment due to their incompatibility with the existing Ethernet compatible infrastructure. Similarly, Ethernet has traditionally not been considered a SAN interconnect due to its close to order-of-magnitude performance gap compared to other SAN interconnects such as InfiniBand, Myrinet and Quadrics (informally called Ethernot networks). However, recently there has been a significant push from both ends of the Ethernet-Ethernot spectrum leading to a convergence of Ethernet and Ethernot. For example, Ethernot networks such as Myrinet and Quadrics are attempting work-arounds in their technologies to allow Ethernet and hence WAN compatibility. Ethernet, on the other hand, is focusing on reaching the performance capabilities of Ethernot networks by utilizing the recently introduced 10-Gigabit Ethernet (10GigE) network and providing hardware offloaded protocol stacks (in particular hardware offloaded TCP/IP or TOE) similar to the other Ethernot networks. In this paper, we focus on the later approach and try to study the potential of 10GigE TOEs in bridging the Ethernet-Ethernot performance gap; specifically compared to the InfiniBand and Myrinet networks. We carry out this evaluation in two stages. First, we compare the performance of the host-based TCP/IP stack over 10GigE to that of 10GigE TOEs in order to understand the performance gains achievable through the use of hardware offloaded protocol stacks for 10GigE. Next, we compar