In-band crosstalk can pose important limitations in an all-optical
wavelength-division-multiplexed (WDM) network. Recent studies have
demonstrated that differential phase shift keying (DPSK), can tolerate
higher in-band crosstalk-noise levels compared to amplitude shift keying
(ASK). In this paper, the performance of a DPSK receiver, limited by
in-band crosstalk noise, is studied theoretically. The model takes into
account both the in-band crosstalk noise as well as the
amplified-spontaneous-emission (ASE) noise of the optical amplifiers.
The model is based on the evaluation of the moment-generating function
(MGF) of the decision variable through which, the error probability (EP)
can be calculated by applying the saddle point approximation. This
provides a rigorous model for the evaluation of the EP of a DPSK
receiver in the presence of ASE and in-band crosstalk noises. In the
absence of the ASE noise, a closed-form formula for the EP is also given
that is useful for estimating the error floor set by the in-band
crosstalk noise
