This chapter overviews the relatively large body of work (experimental and theoretical) on modulation formats for optical coherent links. It first gives basic definitions and performance metrics for modulation formats that are common in the literature. Then, the chapter discusses optimization of modulation formats in coded systems. It distinguishes between three cases, depending on the type of decoder employed, which pose quite different requirements on the choice of modulation format. The three cases are soft-decision decoding, hard-decision decoding, and iterative decoding, which loosely correspond to weak, medium, and strong coding, respectively. The chapter also discusses the realizations of the transmitter and transmission link properties and the receiver algorithms, including DSP and decoding. It further explains how to simply determine the transmitted symbol from the received 4D vector, without resorting to a full search of the Euclidean distances to all points in the whole constellation
