Brennan (Brennan, R. L., 2012) noted that users of test scores often want (indeed, demand) that subscores be reported, along with total test scores, for diagnostic purposes. Haberman (Haberman, S. J., 2008) suggested a method based on classical test theory (CTT) to determine if subscores have added value over the total score. According to this…

The measurement community needs to better understand how to interact with the media to effectively disseminate important findings from educational testing efforts. To this end, the current paper will review media coverage of educational testing and related issues and elaborate on areas of concern and opportunities for improved communication…

Neil Dorans has made a career of advocating for the examinee. He continues to do so in his NCME career award address, providing a thought-provoking commentary on some current trends in educational measurement that could potentially affect the integrity of test scores. Concerns expressed in the address call attention to a conundrum that faces…

Views on testing--its purpose and uses and how its data are analyzed--are related to one's perspective on test takers. Test takers can be viewed as learners, examinees, or contestants. I briefly discuss the perspective of test takers as learners. I maintain that much of psychometrics views test takers as examinees. I discuss test takers as a…

This paper illustrates that the psychometric properties of scores and scales that are used with mixed-format educational tests can impact the use and interpretation of the scores that are reported to examinees. Psychometric properties that include reliability and conditional standard errors of measurement are considered in this paper. The focus is…

The authors describe the significant contributions of Robert Ebel to educational measurement theory and its applications. A biographical sketch details Ebel's roots and professional resume. His influence on classroom assessment views and procedures are explored. Classic publications associated with validity, reliability, and score interpretation…

Although it is assumed that student cognition contributes to student performance on achievement tests, it may be that current testing models lack the degree of specification necessary to warrant such inferences. With test score interpretations as the referent, the authors in this special issue address the role of student cognition in learning and…

Describes the procedures by which Kentucky's state assessment program synthesized results from three standard setting procedures (Contrasting Groups, Bookmark, and Jaeger-Mills) for the 2000 state assessment. Shows the value of using multiple standard-setting approaches to gather information from each. (SLD)

Studied the validity of interpretations derived from school-level assessment scores for different content areas using school-level data from 14 large-scale assessment programs. Results raise suspicions about the value and validity of interpretations based on school-level subject area scores. (SLD)

Describes an approach to addressing issues of validity of inferences and the extrapolation of inferences to target domains beyond the assessment for alternative assessments. Makes the case that in both language testing and educational assessment the roles of language and content knowledge must be considered, and that the design and development of…

Analyzes the three major inferences involved in interpretation of performance assessments: (1) scoring of the observed performances; (2) generalization to a domain of assessment performances like those included in the assessment; and (3) extrapolation to the large performance domain of interest. Suggests ways to improve the validity of performance…

This essay argues that reliability should be reconceptualized in a way that reflects the importance of the theoretical expectations of the test specialist and the learning and problem solving of the test takers. It is time to characterize clearly the substantive theoretical framework supporting reliability studies and the technical evaluation of…

This module prepares the reader to use statistical procedures to detect differentially functioning test items. The Mantel-Haenszel statistic, logistic regression, the SIBTEST procedure, the Standardization procedure, and various item response theory-based procedures are presented. Theoretical frameworks, strengths and weaknesses, and…

It is argued that consequential validity is a concept worth considering. The solution to defining "validity" is not to narrow the concept, but to allow for the differential prediction provided by tests in different circumstances. Consequences of the uses and interpretations of test scores are central to their evaluation. (SLD)

The development and validation of professional development resource materials for teachers by the National Council on Measurement in Education and associated professional groups is described. The developed module contains a context-orienting vignette, instruction, practice exercises, hands-on experiences, small group topics, and homework problems.…