This article reviews research on (a) autobiographical episodic and semantic memory, (b) the self-reference effect, (c) memory for the actions of self versus other (the self-enactment effect), and (d) non-autobiographical episodic memory in autism spectrum disorder (ASD), and provides a theoretical framework to account for the bidirectional relationship between memory and the self in ASD. It is argued that individuals with ASD have diminished psychological self-knowledge (as a consequence of diagnostic social and communication impairments), alongside intact physical self-knowledge, resulting in an under-elaborated self-concept. Consequently, individuals with ASD show impaired autobiographical episodic memory and a reduced self-reference effect (which may each rely on psychological aspects of the self-concept) but do not show specific impairments in memory for their own rather than others' actions (which may rely on physical aspects of the self-concept). However, it is also argued that memory impairments in ASD (e.g., in non-autobiographical episodic memory) may not be entirely accounted for in terms of self-related processes. Other factors, such as deficits in memory binding, may also play a role. Finally, it is argued that deficits in autobiographical episodic memory and future thinking may result in a diminished temporally extended self-concept in ASD. (Contains 3 notes.)