The establishment of the symbiosis between legumes and nitrogen-fixing rhizobia is finely regulated at the transcriptional, posttranscriptional and posttranslational levels. Argonaute5 (AGO5), a protein involved in RNA silencing, can bind both viral RNAs and microRNAs to control plant-microbe interactions and plant physiology. For instance, AGO5 regulates the systemic resistance of Arabidopsis against Potato Virus X as well as the pigmentation of soybean (Glycine max) seeds. Here, we show that AGO5 is also playing a central role in legume nodulation based on its preferential expression in common bean (Phaseolus vulgaris) and soybean roots and nodules. We also report that the expression of AGO5 is induced after 1 h of inoculation with rhizobia. Down-regulation of AGO5 gene in P. vulgaris and G. max causes diminished root hair curling, reduces nodule formation and interferes with the induction of three critical symbiotic genes: Nuclear Factor Y-B (NF-YB), Nodule Inception (NIN) and Flotillin2 (FLOT2). Our findings provide evidence that the common bean and soybean AGO5 genes play an essential role in the establishment of the symbiosis with rhizobia