We report the experimental observation and numerical simulations of a polarization instability of spatial vector solitons in an AlGaAs slab waveguide. At power levels where the nonlinear index change becomes comparable to the birefringence, the fast soliton becomes unstable. The instability is related to coupling of the fast soliton to the slow radiation modes through phase matching. The combined effects of bifurcation and radiation coupling are the processes ultimately limiting the stability of any single-polarization (fast and slow) Kerr soliton