Unlike adult memories that can be remembered for many years, memories that are formed early in life are more fragile and susceptible to being forgotten (a phenomenon known as "infantile" or "childhood" amnesia). Nonetheless, decades of research in both humans and nonhuman animals demonstrate the importance of early life experiences on later physical, mental, and emotional functioning. This raises the question of how early memories can be so influential if they cannot be recalled. This review presents one potential solution to this paradox by considering what happens to an early memory after it has been forgotten. Specifically, we describe evidence showing that these forgotten early-acquired memories have not permanently decayed from storage. Instead, there appears to be a memory "trace" that persists in the face of forgetting which continues to affect a variety of behavioral responses later in life. Excitingly, the discovery of this physical trace will allow us to explore previously untestable issues in new ways, from whether forgetting is due to a failure in retrieval or storage to how memories can be recovered after extended periods of time. A greater understanding of the characteristics of this memory trace will provide novel insights into how some memories are left behind in childhood while others are carried with us, at least in some form, for a lifetime.