We present constraints on the flat Λ cold dark matter cosmological model through a joint analysis of galaxy abundance, galaxy clustering, and galaxy-galaxy lensing observables with the Kilo-Degree Survey. Our theoretical model combines a flexible conditional stellar mass function, which describes the galaxy-halo connection, with a cosmological N-body simulation-calibrated halo model, which describes the non-linear matter field. Our magnitude-limited bright galaxy sample combines nine-band optical-to-near-infrared photometry with an extensive and complete spectroscopic training sample to provide accurate redshift and stellar mass estimates. Our faint galaxy sample provides a background of accurately calibrated lensing measurements. We constrain the structure growth parameter to S8 = σ8√Ωm/0.3 =√0.773−0.030+0.028 and the matter density parameter to Ωm = 0.290−0.017+0.021. The galaxy-halo connection model adopted in the work is shown to be in agreement with previous studies. Our constraints on cosmological parameters are comparable to, and consistent with, joint ‘3 × 2pt’ clustering-lensing analyses that additionally include a cosmic shear observable. This analysis therefore brings attention to the significant constraining power in the often excluded non-linear scales for galaxy clustering and galaxy-galaxy lensing observables. By adopting a theoretical model that accounts for non-linear halo bias, halo exclusion, scale-dependent galaxy bias, and the impact of baryon feedback, this work demonstrates the potential for, and a way towards, including non-linear scales in cosmological analyses. Varying the width of the satellite galaxy distribution with an additional parameter yields a strong preference for sub-Poissonian variance, improving the goodness of fit by 0.18 in terms of the reduced χ2 value (and increasing the p-value by 0.25) compared to a fixed Poisson distribution