On the basis of a validation study of a new test for assessing low-achieving adolescents' reading comprehension skills--the SALT-reading--we analyzed two issues relevant to the field of reading test development. Using the test results of 200 seventh graders, we examined the possibility of identifying reading comprehension subskills and the effects of task specificity on test reliability. Regarding the former, we distinguished three subskills indicating different levels of understanding ("retrieving", "interpreting", "reflecting"). However, confirmatory factor analyses did not support the presence of these subskills. Task specificity refers to the situation that different tasks within a test are not uniformly difficult for individual test takers, which constitutes a form of error negatively influencing test reliability. However, Generalizability Theory analysis showed that such task-specific effects did not occur: the reliability of the SALT-reading was primarily affected by error associated with the score variance within tasks. (Contains 4 tables, 3 figures and 2 notes.)