We investigate the emergence of in-phase synchronization in a heterogeneous network of coupled inhibitory interneurons in the presence of spike timing dependent plasticity (STDP). Using a simple network of two mutually coupled interneurons (2-MCI), we first study the effects of STDP on in-phase synchronization. We demonstrate that, with STDP, the 2-MCI network can evolve to either a state of stable 1:1 in-phase synchronization or exhibit multiple regimes of higher order synchronization states. We show that the emergence of synchronization induces a structural asymmetry in the 2-MCI network such that the synapses onto the high frequency firing neurons are potentiated, while those onto the low frequency firing neurons are de-potentiated, resulting in the directed flow of information from low frequency firing neurons to high frequency firing neurons. Finally, we demonstrate that the principal findings from our analysis of the 2-MCI network contribute to the emergence of robust synchronization in the Wang-Buzsaki network (Wang and Buzsáki, 1996) of all-to-all coupled inhibitory interneurons (100-MCI) for a significantly larger range of heterogeneity in the intrinsic firing rate of the neurons in the network. We conclude that STDP of inhibitory synapses provide a viable mechanism for robust neural synchronization