Language teachers often encourage L2 readers to guess the meaning of unknown words in texts. This paper argues that before any statements about the value of guessing can be made, it is necessary to show how reliable learners' guesses are. To this end, the present study investigates how accurately EFL learners assess their understanding of unknown words, and whether language proficiency and gender influence their self-assessment. The study was carried out in three phases. In phase one, participants (N = 110) took a proficiency test and were asked to study a text in which 40 items were highlighted by the researcher on the basis of words that were unfamiliar to them. They were required to assess their understanding of the meaning of unknown words on a four-point scale. In phase two, they provided a translation, synonym, antonym, or an explanation of the unknown words. In phase three, they received the same self-assessment sheet to reassess their understanding of unknown words. The results indicated that learners' assessments are not reliable. Moreover, it was found that proficiency in the L2 and gender influence learners' self-assessment. (Contains 6 tables.)