Optimal constellations design for trellis coded unitary space-time modulation

We give a new formulation of the problem of signal design based on the design criteria for trellis coded unitary space-time communication system. Two key differences in our approach from that of previous method are that we not only combine the constellation design and set partition process together, we also use directly the Chernoff bound of the pairwise probability of error as design metric, which is a better figure of merit than previously used chordal distance, for unitary constellation design. By novelly employing a theorem for the Clarke subdifferential of the sum of the k largest singular values of the unitary matrix, we present a numerical optimization procedure for finding good constellations and report the best signal constellations found. Simulation results show that these constellations improve significantly upon the previously used constellations of unitary space-time signals. © 2005 IEEE