We propose amplitude caching to optically equalise burst mode traffic without delay stages. Through a fast, optically-switched system prototype, we demonstrate burst-mode penalties can be mitigated to within 0.4 dB at the KR4 HD-FEC level

Ballani, H

Clark, K

Costa, P

Funnell, A

Gerard, T

Haller, I

Jozwik, K

Shi, K

Thomsen, B

Watts, P

Williams, H

English

UCL Discovery

Fast and Uniform Optically-Switched DataCentre Networks Enabled by Amplitude CachingThomas Gerard1,*, Kari Clark1, Adam Funnell2, Kai Shi3, Benn Thomsen3, Philip Watts1, KrzysztofJozwik3, Istvan Haller3, Hugh Williams3, Paolo Costa3 and Hitesh Ballani3(1) Optical Networks Group, Dept. Electronic & Electrical Engineering, University College London, London, WC1E 7JE, UK.(2) Multidisciplinary Engineering Education, University of Sheffield, 32 Leavygreave Rd, Sheffield, S3 7RD, UK.(3) Microsoft Research, 21 Station Road, Cambridge, CB1 2FB, UK.*uceetmh@ucl.ac.ukAbstract: We propose amplitude caching to optically equalise burst mode traffic withoutdelay stages. Through a fast, optically-switched system prototype, we demonstrate burst-mode penalties can be mitigated to within 0.4 dB at the KR4 HD-FEC level. © 2021 TheAuthor(s)1. IntroductionData centres are fast approaching a networking bottleneck, where their incumbent electrically-switched networksmay struggle to keep pace with the exponentially growing demand for bandwidth. This looming crunch has re-newed interest in fast optical circuit switching (OCS), which can be used to create flat, energy-efficient, low-latency networks [1, 2]. A wide variety of OCS architectures have been proposed based on wavelength routing,space switching, or a combination thereof [2], but all OCS systems face the common challenges of varying opticalloss and phase across the network paths. These effects can introduce baseline wander and symbol-missampling atthe receiver and are particularly hard to account for in sub-microsecond granularity switching systems. However,the data centre represents a unique, closed environment in which the loss and phase changes between source-destination pairs are relatively stable; hence, instead of discovering the phase and gain offsets through complexcircuits at burst-granularity, they can be remembered or cached and applied at either end. Previously, we havedemonstrated that fast clock phase recovery can be achieved by creating a synchronous data centre network thatapplies clock phase caching (CPC) to phase-align all transceivers with sub-bit precision [6]. In this work, weextend this principle to ‘amplitude caching’, in which optical equalisation is applied prior to transmission (or re-ception) to mitigate baseline wander in fast burst-mode transceivers. We combine amplitude and phase caching ina generic space-and wavelength switched testbed to create a power- and phase-uniform OCS system, for the firsttime. By testing our system with real time 25 Gbps on-off keying (OOK) burst data, we demonstrate low-penaltyburst switching that permits the creation of highly scalable, flat, energy efficient, OCS data centre networks.2. Concept of Amplitude CachingPreviously, optical equalisers have been proposed that use feed-forward semiconductor optical amplifier (SOA)gain control circuits to normalise the received optical power of burst-mode detectors [5,6]. However, these methodsrequire additional SOAs, burst headers and delay stages that are incompatible with the fast amplitude recoveryrequired by bursty data centre traffic. In this work, we exploit the deterministic path-loss of a flat data centrenetwork to introduce ‘amplitude caching’, in which optical equalisation is applied at the transmitter. The principleof operation is shown in Fig. 1 which considers an N node network intra-connected through an N ×N opticalswitch. Each node’s optical transmitter is followed by a device capable of fast optical power control - our prototypeuses SOAs, commonly included in optical transmitters, though fast optical attenuators or modulators could alsobe used. A two-stage process is used to achieve optical equalisation. The calibration stage is shown in Fig. 1(a),in which the equalisation of Node 0 is considered. Nodes 1 to N consecutively transmit dummy-data to Node 0,which measures the varying received optical powers. Node 0 then transmits feedback to Nodes 1 to N, informingthem by how they should attenuate their output to match the received burst with the lowest power. Nodes 1 to Nlog this information in a (N-1) entry lookup table (LUT). This process is repeated for all nodes in the networkuntil all LUTs are populated. The operation stage is shown in Fig. 1(b). When transmitting to Node 0, Nodes 1to N use the cached amplitude values to vary the current applied to the SOA. This compensates for the varyingpath loss across the network path and the optical switch, and guarantees uniform-intensity burst-to-burst receptionat Node 0. Optical path loss is not expected to vary significantly over time; however, the amplitude cache valuescan be periodically updated using in-band [1] or out-of-band [3] communication schemes. Amplitude caching,therefore, represents a low complexity method for achieving fast optical equalisation throughout a data centrenetwork without requiring complex receiver-side gain control circuits or delay stages.feedbackN×Nopticalnetwork(a) CalibrationSOARxTxLUTLUTTxRxSOANode 0feedbackNode 1LUTupdateNode 2Node NN×Nopticalnetwork(b) OperationSOARxTxLUTLUTTxRxSOANode 0Node 1LUTactiveNode 2Node NLUTactivevariableinputsuniformoutputsFig. 1. Concept of amplitude caching applied to a flat N ×N optical network. (a) The optical loss of all intra-connections are pre-calibrated and cached in a lookup table (LUT) in each node. (b) During normal operation,nodes vary their launch power using the LUTs to optically equalise the received signal.3. Experimental SetupAmplitude caching is a general technique that can be used in both space and/or wavelength switched networks;to demonstrate this, our setup uses a combination of both. Two space- and wavelength-switching nodes, NodeA and Node B, were constructed using off-the-shelf components as shown in Fig. 2(a), and arranged into thethe experimental setup shown in Fig. 2(b). A field programmable gate array (FPGA) was used to generate andreceive real time data and act as a unified control-hub to the nodes’ optical subsystems. Each node contained asemiconductor tuneable laser (TL) supporting in excess of 96 C-band 50 GHz wavelength channels, modulatedby a 20 GHz intensity modulator. The modulator was biased at the quadrature point and driven with RF-amplified25 Gb/s OOK data using the FPGA’s inbuilt transmitter (TxA for Node A and TxB for Node B). An opticalbroadcast and select (OB&S) stage was emulated using an optical power coupler with outputs passed to twodiscrete semiconductor optical amplifiers (SOAs) supporting 69 nm of bandwidth with typical characteristics of7 dB noise figure, 20 dB gain, and 10 dBm saturation power. These SOAs were used for both space-switchingand amplitude caching. Space switching was achieved by driving one SOA in the ‘on’ state while the other wasdriven with a small negative current, creating an ‘off’ state with > 30 dB extinction. Greater OB&S orders wereemulated using a variable optical attenuator (VOA) before the optical coupler. The four OB&S outputs (NodeA SOA0 and SOA1, and Node B SOA0 and SOA1) were passed to a 32×32 arrayed waveguide grating router(AWGR) for wavelength routing. TL wavelengths and AWGR input ports were selected to ensure the four inputlines mapped to just two optical AWGR outputs, where were each connected to an avalanche photodiode (APD)with typical responsivity of 2 A/W in the C-band. The APD output was passed to the FPGA receiver (RxA forNode A and RxB for Node B), which performed real time sampling with an adaptive CDR circuit before makinga hard decision on the received data. The measured bits were downloaded for offline analysis in MATLAB.4. Results and DiscussionSpace- and wavelength optical burst switching was demonstrated by setting each node’s FPGA to instruct theTL and SOAs to play out a repeating train of four optical bursts. Each burst was made up of a 29 − 1 pseudo-random bit sequence (PRBS) played out on repeat to emulate a 4 kB packet persisting for 1.3 µs at 25 Gb/s.Between each burst, the SOAs were set to the ‘off’ state, the TL wavelengths reconfigured, and the new SOAsset to ‘on’, requiring a last-bit to first-bit guard band of 42 ns. This can be reduced to 3.84 ns by using noveltuneable sources [4]. Fig. 2(c)(i) shows the output of the APD in Node B while receiving four sequential opticalbursts from both nodes, measured using an 80 GS/s oscilloscope. The received optical power varies significantlyfrom burst to burst; this is attributed to wavelength-dependent TL power, non-ideal coupling losses, and variableTL ModSOA0SOA1APDRxAFPGATxACTRLλNode ATL ModSOA0SOA1APDRxBFPGATxBCTRLλNode BAWGROB&SCTRLCTRLOB&SLaserSOAsFPGA MZMMod.DriverAPD(a) (b)Baseline wanderSteady baselineNoAmp.cacheWithAmp.cache(i)(ii)(c)Fig. 2. (a) Space- and wavelength-switching node. (b) Two nodes perform fast OCS connections by switchingthe TL wavelength and active SOA. (c) APD output in Node B (i) without and (ii) with amplitude (Amp.)caching applied. Each burst represents a unique wavelength/SOA state from both TxA and TxB.−π0π(a)CDRphase No CPC With CPC Bit errors0 1 2 3 4 5−π0π(b)Bit index / 105CDRphase−23 −22 −21 −20 −19 −1810−710−610−510−410−310−2HD-FEC 0.4 dB(c)Received Power (dBm)BERContinuousBurst, no CPCBurst, with CPCFig. 3. Adaptive CDR phase in RxA when receiving burst mode data (a) without, and (b) with clock phasecaching (CPC) enabled. Bit error locations are shown as crosses. (c) Associated BER curves.AWGR insertion loss. The power variation is seen to introduce baseline wander to the AC-coupled APD output.To correct this, we apply amplitude caching as described in section 2 using the ‘on’ SOA in each node’s OB&Sstage for fast optical equalisation. The APD output with amplitude caching applied is given in Fig. 2(c)(ii), whichshows that the burst-to-burst power variation have been suppressed to < 0.5 dB and that baseline wander has beenremoved. For the switch combinations tested in the work, up to a 2 dB reduction in transmitter launch power wasrequired to correct for varying path loss, chiefly caused by variable AWGR insertion loss.With amplitude caching applied, the burst mode data was received using the real time FPGA receivers. Theinbuilt CDR circuits were used to adaptively adjust the real time sampler, correcting for phase-misalignments be-tween TxA and TxB. Fig. 3(a) shows the adaptive CDR phase of RxA across 15 bursts, which is observed to makesudden, significant updates between bursts. This is because the FPGA transmitters are not phase matched and,therefore, the CDR unit in RxA must correct for phase misalignment after the burst arrival. Complete alignmentcan take 100’s of ns to complete, and during this period non-ideal sample timing can introduce bit errors. Thelocation of measured bit errors are overlaid on Fig. 3(a), and show a strong correlation with locations in whichlarge CDR phase updates are required. To create a uniform phase response at each receiver we apply clock phasecaching (CPC) [3], which exploits the FPGA’s ability to phase shift the transmitted data prior to modulation. Aswith amplitude caching, the FPGA in each node was provided with an (N-1)-entry lookup table that contains thephase offsets required to correct for the misalignment associated with every receiver in the network. CPC wasturned on for both TxA and TxB and the CDR phase remeasured at RxA. Fig. 3(b) shows that, with CPC on,phase variance has been reduced by > 20 dB and the associated bit errors have been removed.The advantage of a uniform power-and phase OCS system is shown by the bit error rate (BER) curves inFig. 3(c). Here we target a BER better than 5×10−5 for use with the KR4 Reed-Solomon hard decision forwarderror correction code (HD-FEC), overhead 2.7%, widely used in coded optical data centre systems. The datawithout phase caching exhibits a noise floor around a BER of 1× 10−4, preventing error free recovery duringburst mode operation. With amplitude and phase caching applied this noise floor is removed, allowing our systemto achieve low BERs better than 1× 10−6 (limited by our FPGA’s capture size). HD-FEC ready performance isachieved at an average received optical power of −20.7 dBm, a penalty of just 0.4 dB with respect to our systemwhen transmitting continuous data (TxA transmitting on 1560.67 nm through SOA0 to RxA). This result showsthat the application of amplitude and phase caching eliminates the bandwidth limitations of receiver-side recoverymethods, enabling low-penalty OCS. CPC has been shown to fully mitigate burst errors to a BER < 1× 10−12;therefore, the 0.4 dB of penalty observed here is attributed to residual SOA intensity variation on the rising edge.This can be corrected using intelligent SOA drive systems [7].5. ConclusionsIn this system demonstration, we apply amplitude and clock phase caching to create, for the first time, a power-and phase-uniform fast OCS system. The transmitter-side techniques are shown to push error floors to below aBER of 10−6, mitigating real-time burst mode penalties to within 0.4 dB at the KR4 HD-FEC level. These generaltechniques can benefit any optically switched system, and can be used to create scalable flat data centre networks.This work was supported by TRANSNET (EP/R035342/1) and Microsoft Research through its PhD scholarship programme. The authorsthank Prof. P. Bayvel for useful discussion.References1. H. Ballani et al., “Sirius: A Flat Datacenter Network with Nanosecond Optical Switching”, Proc. SIGCOMM, 2020.2. Q. Cheng. et al., “Photonic switching in high performance datacenters”, OE 26(12), 2018.3. K. Clark et al., “Synchronous subnanosecond clock and data recovery for optically switched data centres using clock phase caching”,Nature Electronics 3, 426–433, 2020.4. K. Shi et al., “System demonstration of nanosecond wavelength switching with burst-mode PAM4 transceiver”, ECOC pdp, 2019.5. E. Aw et al., “Layered control to enable large scale SOA switch fabric”, Proc. ECOC, Th1.2.5, 2006.6. B. Thomsen et al., “Optically equalised 10Gb/s NRZ digital burst-mode receiver for dynamic optical networks”, OE 15(15), 2007.7. C. Parsonson et al., “Optimal control of SOAs With artificial intelligence for sub-nanosecond optical switching”, JLT 38(20), 2020.

Fast and Uniform Optically-Switched Data Centre Networks Enabled by Amplitude Caching

https://discovery.ucl.ac.uk/10133406/1/gerard21fast.pdf

Fast and Uniform Optically-Switched Data Centre Networks Enabled by Amplitude Caching

Abstract

Similar works

Full text

Available Versions

UCL Discovery