This piece is framed by questions we are often asked when we talk about online reading assessments and instruction with teachers. We begin with some of the lessons we have learned in our own experiences with designing measures of online reading comprehension. Then we share our thoughts about key design considerations as well as some of the biggest challenges of trying to capture online research and reading skills when online texts and authentic tasks are constantly changing. By sharing these perspectives, we hope to inspire new ways of tracking students' progress that can help teachers design powerful literacy instruction that integrates digital texts and new technologies.