The purpose of this study was to examine the technical adequacy of a computer-based assessment instrument which is based on hierarchical models of text comprehension for monitoring student reading progress following the Curriculum-Based Measurement (CBM) approach. At intervals of two weeks, 120 third-grade students finished eight CBM tests. To examine the construct validity, group-administered standardized achievement tests of reading, mathematics, and intelligence were applied. Results indicate that the technical adequacy of the concept is comparable to that of established measures of reading progress. In addition, the new assessment tool provides differentiated diagnostic information about single components of reading proficiency and directly assesses higher processes of comprehension. Benefits of a multidimensional measurement of reading progress for students with special educational needs are discussed. (Contains 6 tables and 2 figures.)