The present paper addresses the continuous need for methodological reflection on how to validate inferences made on the basis of test scores. Validation is a process that requires many lines of evidence. In this article we discuss the potential of eye tracking methodology in process-oriented reading test validation. Methodological considerations are highlighted and special significance is placed on the importance of studying the first reading of a text as well as reading while answering questions about it. This point of view expands the traditional scope of eye-tracking methodology in reading research. We conducted a small-scale study in which 18 12-year olds read and answered questions about a multimodal text. In this study comprehension scores were related to allocation of visual attention in two conditions: (i) reading a text passage for the first time; and (ii) rereading of the text passage while answering questions about it.