The study of song learning and the neural song system has provided an important comparative model system for the study of speech and language acquisition. We describe some recent advances in the bird song system, focusing on the role of off-line processing including sleep in processing sensory information and in guiding developmental song learning. These observations motivate a new model of the organization and role of the sensory memories in vocal learning. (Contains 6 figures.)