The use of learning games within the classroom is becoming increasingly common because of their potential to positively impact learning. Recent developments in adaptivity offer further possibilities to personalise learning by tailoring the game to an individual child's level or particular learning needs. However, designing an adaptive learning game is a complex process as many different game components have an impact on the provision of optimal challenge, crucial for maintaining player engagement, with limited prior work considering the multifaceted nature of this concept. This paper explores how to design for "challenge" within large-scale adaptive learning games through a case study focused on the design of a literacy game for three linguistically and cognitively diverse learner groups--novice readers, children with dyslexia and children learning English as a foreign language. In reflecting on our design process, we identify three key design tensions that arose: (a) supporting longer-term learning goals through game replayability; (b) fostering either replication or innovation in pedagogy through adaptivity rules; and (c) addressing diversity between learner groups. We present a set of design recommendations to guide researchers and designers in taking a multidimensional view of challenge when designing large-scale adaptive learning games.