People use language to exchange ideas and influence the actions of others through shared conceptions of word meanings, and through a shared understanding of how word meanings are combined. Under the surface form of words lie complex networks of mental structures and processes that give rise to the richly textured semantics of natural language. Machines, in contrast, are unable to use language in human-like ways due to fundamental limitations of current computational approaches to semantic representation. To address these limitations, and to serve as a catalyst for exploring alternative approaches to language and meaning, we are developing conversational robots. The problem of endowing robots with language highlights the impossibility of isolating language from other cognitive processes. Instead, we embrace a holistic approach in which various non-linguistic elements of perception, action, and memory, provide the foundations for grounding word meaning. I will review recent results in grounding language in perception and action and sketch ongoing work for grounding a wider range of words including social terms such as "I" and "my"