This article presents a framework for understanding historical perspective taking (HPT), the effort to use historical material to explore the internal states of past people. It addresses gaps in HPT research by (a) linking HPT to theories and research from the social science disciplines on perspective taking and the self and (b) proposing a way to analyze how different tasks may elicit different kinds of HPT. The framework is grounded in a study in which 4 young adults thought aloud while taking the perspective of a victim or a perpetrator at the Salem Witch Trials and in a Holocaust-era massacre. Four shifting self perspectives were identified, whereby participants thought as their present-day selves, constructed hypothetical and imagined past selves, and made timeless generalizations about humans. To demonstrate the utility of the framework, I then use it to consider how task characteristics may elicit differential use of the self perspectives. It is argued that close attention to these self perspectives is a novel and crucial way to bring nuance to the concept of HPT, and implications for multiple learning environments are discussed.