We discuss a research-based theoretical framework based on affect as an internal representational system. Key ideas include the concepts of meta-affect and affective structures, and the constructs of mathematical intimacy and mathematical integrity. We understand these as fundamental to powerful mathematical problem solving, and deserving of closer attention by educators. In a study of elementary school children we characterize some features of emotional states inferred from individual problem solving behavior, including interactions between affect and cognition. We describe our methodology, illustrating theoretical ideas with brief qualitative examples from a longitudinal series of task-based interviews.