A standard load b lancing model considers placing n b lls into nb[K y choosing d possib: locations for eachb all independently and uniformly at random and sequentially placing each in the least loaded of its chosenb ins. It is well known that allowing just a small amount of choice (d = 2) greatly improves performance over random placement (d = 1). In this paper, we show that similar performance gains occurb y introducing memory. We focus on the situation where each time ab all is placed, the least loaded of that b ll's choices after placement is rememb ered and used as one of the possibH choices for the nextb all. For example, we show that when eachb all gets just one random choice,be can also choose theb est of the lastb all's choices, the maximum numb er ofb alls in ab in is log log n/2log# +O(1) with high pro bbU:: y, where # =(1+ # 5)/2 is the golden ratio. The asymptotic performance is thereforeb etter with one random choice and one choice from memory than with two fresh random choices for eachb all; the performance with memory asymptotically matches the asymmetric policy using two choices introducedb y Vocking. More generally, we find that a small amount of memory, like a small amount of choice, can dramatically improve the loadb alancing performance. We also investigate continuous time variations corresponding to queueing systems, where we find similar results.