Background: Human pluripotent stem cells offer the best available model to study the underlying cellular and molecular mechanisms of human embryonic lineage specification. However, it is not fully understood how individual stem cells exit the pluripotent state and transition towards their respective progenitor states. Results: Here, we analyze the transcriptomes of human embryonic stem cell-derived lineage-specific progenitors by single-cell RNA-sequencing (scRNA-seq). We identify a definitive endoderm (DE) transcriptomic signature that leads us to pinpoint a critical time window when DE differentiation is enhanced by hypoxia. The molecular mechanisms governing the emergence of DE are further examined by time course scRNA-seq experiments, employing two new statistical tools to identify stage-specific genes over time (SCPattern) and to reconstruct the differentiation trajectory from the pluripotent state through mesendoderm to DE (Wave-Crest). Importantly, presumptive DE cells can be detected during the transitory phase from Brachyury (T)+ mesendoderm toward a CXCR4+ DE state. Novel regulators are identified within this time window and are functionally validated on a screening platform with a T-2A-EGFP knock-in reporter engineered by CRISPR/Cas9. Through loss-of-function and gain-of-function experiments, we demonstrate that KLF8 plays a pivotal role modulating mesendoderm to DE differentiation. Conclusions: We report the analysis of 1776 cells by scRNA-seq covering distinct human embryonic stem cell-derived progenitor states. By reconstructing a differentiation trajectory at single-cell resolution, novel regulators of the mesendoderm transition to DE are elucidated and validated. Our strategy of combining single-cell analysis and genetic approaches can be applied to uncover novel regulators governing cell fate decisions in a variety of systems. Electronic supplementary material The online version of this article (doi:10.1186/s13059-016-1033-x) contains supplementary material, which is available to authorized users