Early embryonic cells in many species polarize radially by distinguishing their contacted and contact-free surfaces. Radial polarization is a critical patterning event driven by cell-cell contact and is required for developmental processes, such as the first differentiation event in the early mammalian embryo. The homophilic adhesion protein E-cadherin is required for contact-induced polarity in many cells. However, it is not clear whether E-cadherin functions instructively as a spatial cue, or permissively by ensuring adequate adhesion so that cells can sense other contact signals. In C. elegans, radial polarity begins at the four-cell stage, when cell contacts restrict the PAR polarity proteins to contact-free surfaces. We previously identified the RhoGAP PAC-1 as an upstream regulator that is required to exclude PAR proteins from contacted surfaces of early embryonic cells. PAC-1 is recruited specifically to sites of cell contact and directs PAR protein asymmetries by inhibiting the Rho GTPase CDC-42. How PAC-1 is able to sense where contacts are located and localize to these sites is unknown. We show that HMR-1/E-cadherin, which is dispensable for adhesion, functions together with HMP-1/α-catenin, JAC-1/p120 catenin, and the previously uncharacterized linker PICC-1/CCDC85/DIPA to bind PAC-1 and recruit it to contacts. Furthermore, we show that ectopically localizing the intracellular domain of HMR-1/E-cadherin to contact-free surfaces of cells recruits PAC-1 and depolarizes cells, demonstrating that HMR-1/E-cadherin plays an instructive role in polarization. Furthermore, we show that radial polarity is defective in embryos lacking HMR-1/E-cadherin. Our findings identify an E-cadherin-mediated pathway that translates cell contacts into cortical polarity by directly recruiting a symmetry-breaking factor to the adjacent cortex