It is well established observationally that the characteristic angular momentum axis on small scales around active galactic nuclei (AGN), traced by radio jets and the putative torus, is not well correlated with the large-scale angular momentum axis of the host galaxy. In this paper, we show that such misalignments arise naturally in high-resolution simulations in which we follow angular momentum transport and inflows from galaxy to sub-pc scales near AGN, triggered either during galaxy mergers or by instabilities in isolated discs. Sudden misalignments can sometimes be caused by single massive clumps falling into the centre slightly off-axis, but more generally, they arise even when the gas inflows are smooth and trace only global gravitational instabilities. When several nested, self-gravitating modes are present, the inner ones can precess and tumble in the potential of the outer modes. Resonant angular momentum exchange can flip or re-align the spin of an inner mode on a short time-scale, even without the presence of massive clumps. We therefore do not expect that AGN and their host galaxies will be preferentially aligned, nor should the relative alignment be an indicator of the AGN fuelling mechanism. We discuss implications of this conclusion for AGN feedback and black hole (BH) spin evolution. The misalignments may mean that even BHs accreting from smooth large-scale discs will not be spun up to maximal rotation and so have more modest radiative efficiencies and inefficient jet formation. Even more random orientations/lower spins are possible if there is further unresolved clumpiness in the gas, and more ordered accretion may occur if the inflow is slower and not self-gravitating
