The present study examined the reliability of an achievement test to measure the efficacy of task-based writing activities to improve Iranian EFL learners' reading comprehension at the intermediate level in a private language institute in Ilam, Iran, namely Alefba language institute. To achieve the goal, the techniques for evaluating reliability of criterion-referenced tests proposed by Brown and Hudson (2002) were employed. To calculate the reliability of the developed test, or using Brown and Hudson's (2002) term "dependability," two coefficients of agreement (?o) and Kappa (?) were computed following the appropriate way Brown and Hudson have offered. The results demonstrate that the developed achievement test is a reliable measure to assess the efficacy of task-based writing activities to advance the selected participants' reading comprehension skill. Implications of present findings and suggestions for further research are discussed as well.