Advancing learners' agency is a key educational goal. The advent of personalized EdTech, which automatically tailor learning environments to individual learners, gives renewed relevance to the topic. EdTech researchers and practitioners are confronted with the same basic question: What is the right amount of agency to give to learners during their interactions with EdTech? This question is even more relevant for younger learners. Our aim in this paper is twofold: First, we outline and synthesize the ways in which agency is conceptualized in three key learning disciplines (philosophy, education, and psychology). We show that there are different types and levels of agency and various prerequisites for the effective exercise of agency and that these undergo developmental change. Second, we provide guiding principles for how agency can be designed for in EdTech for children. We propose an agency personalization loop in which the level of agency provided by the EdTech is assigned in an adaptive manner to strike a balance between allowing children to freely choose learning content and assigning optimal content to them. Finally, we highlight some examples from practice.