This paper compares modified Guttman and item response theory (IRT) based procedures for classifying examinees in ordered levels when each level is represented by several multiple choice test items. In the modified Guttman procedure, within-level number correct scores are mapped to binary level mastery scores. Examinees are then assigned to levels in Guttman fashion on the basis of their observed patterns of binary mastery scores over levels. In the IRT procedure, examinees are assigned to levels on the basis of their total number correct score over all items on the test. An IRT-based method is used to assess the reliability and classification consistency of both level scoring procedures for the Work Keys assessment with samples of approximately 2,000 examinees for three forms of both the reading and mathematics assessments. In comparison to the modified Guttman procedure, IRT-based number correct level scores are more reliable and correct across forms. Guttman-consistent patterns of mastery over levels can also be inferred from number correct level scores. (Contains 9 tables, 10 figures, and 34 references.) (Author/SLD)