The immediate written recall task, a widely used measure of both first language (L1) and second language (L2) reading comprehension, has been advocated over traditional test methods such as multiple choice, cloze tests and open-ended questions because it is a direct and integrative assessment task. It has been, however, criticized as requiring memory. Whether and how the requirement of memory biases our understanding of readers' comprehension remains unexplored. This study compares readers' performance on the immediate recall and a translation task in order to explore the effect of memory on readers' recall. Ninety-seven college students participated in this study. All participants were native speakers of Mandarin Chinese whose ages ranged from 20 to 22. The results showed that the translation task yielded significantly more evidence of comprehension than did the immediate recall task, which indicates that the requirement of memory in the recall task hinders test-takers' ability to demonstrate fully their comprehension of the reading passage. The results also showed that the significant difference found in learners' performance between the immediate recall and the translation task spanned the effect of topics and proficiency levels.