Discusses some problems, pitfalls, and paradoxes that challenge measurement theory and practice, especially for K-12 achievement testing. Considers a number of technical issues, especially some related to reliability. Also discusses a number of practical or political issues related to validation and accountability. (SLD)

Responds to a negative evaluation of the National Assessment of Educational Progress (NAEP) by the National Academy of Sciences (NAS) and asserts that a review of the evidence for the NAEP performance standards indicates that there is support for the current approach to NAEP standard setting. Considers the scholarship of the NAS evaluation…

Presents a broad definition of "testlet" and suggests a framework for classifying types of testlets. Considers several issues that bear on the conceptualization of testlets and analyses of scores from tests composed of testlets. Suggests some research topics that seem particularly important to advancing the meaningful and appropriate use of…

Reviews some of the more salient conceptual features of generalizability (G) theory. Provides a framework for considering some misconceptions about G theory. Focuses on particularly prevalent or serious misconceptions. G theory provides a rich conceptual framework and tools for working with a defined universe, but it does not tell an investigator…

Explores the relationship between measurement theory and practice, considering five broad categories of: (1) models, assumptions, and terminology; (2) reliability; (3) validity; (4) scaling; and (5) setting performance standards. It must be recognized that measurement is not an end in itself. (SLD)

The history of generalizability theory (G theory) is told from the perspective of one researcher's experiences, describing psychometric and scientific perspectives that influenced the development of G theory and its adoption. Work that remains to be done in the field is outlined. (SLD)

The framework and procedures of generalizability theory are introduced and illustrated in this instructional module that uses a hypothetical scenario involving writing proficiency. Generalizability analyses are useful for understanding the relative importance of various sources of error and for designing efficient measurement procedures. (SLD)

The application of generalizability theory to the reliability and error variance estimation for performance assessment scores is discussed. Decision makers concerned with performance assessment need to realize the restrictions that limit generalizability such as limitations that lead to reductions in the number of tasks possible, rater quality,…