We carried out a systematic study of magnetic ordering and magnetic interlayer coupling in Ga1-xMnxAs1-yPy/Ga1-xMnxAs bilayers using superconducting quantum interference device magnetometry and ferromagnetic resonance. Such bilayers are interesting, because the easy axis of the constituent materials are orthogonal. Our results show that the bilayers are strongly exchange-coupled at the interface, that manifests itself in the form of horizontal exchange-bias-like shifts of the hysteresis loops of the Ga1-xMnxAs layer, as observed in field-cooled magnetic measurements