The gut microbiota significantly regulates the development and function of the innate and adaptive immune system. The attribute of immunological memory has long been linked only with adaptive immunity. Recent evidence indicates that memory is also present in the innate immune cells such as monocytes/macrophages and natural killer cells. These cells exhibit pattern recognition receptors (PRRs) that recognize microbe- or pathogen-associated molecular patterns (MAMPs or PAMPs) expressed by the microbes. Interaction between PRRs and MAMPs is quite crucial since it triggers the sequence of signaling events and epigenetic rewiring that not only play a cardinal role in modulating the activation and function of the innate cells but also impart a sense of memory response. We discuss here how gut microbiota can influence the generation of innate memory and functional reprogramming of bone marrow progenitors that helps in protection against infections. This article will broaden our current perspective of association between the gut microbiome and innate memory. In the future, this knowledge may pave avenues for development and designing of novel immunotherapies and vaccination strategies