ERIC Number: ED471081
Record Type: RIE
Publication Date: 2002
Generalizability of Performance-Based Assessments. Technical Guidelines for Performance Assessment.
Miller, M. David
In 1994 the State Collaborative on Assessment and Student Standards of the Council of Chief State School Officers began a study to examine the generalizability of performance-based assessments (PBAs) for state-mandated assessment programs. The intent was to examine the major sources of error associated with PBAs and the generalizability and dependability of the assessments. Also examined were the consistency of results of generalizability studies across content areas and the generalizability of student- and school-level data. Generalizability analyses were conducted using state performance-based assessments from four states: Alabama, Connecticut, Delaware, and North Carolina. Most of the assessment programs are using PBAs that have a reasonable level of generalizability (above 0.70) at the student level. However, the level of generalizability is not as high as that found for more traditional forms of assessment. Other advantages of PBAs, such as their consequences for assessment, are necessary to justify their use. Results at the school level are mixed. While the standard errors are smaller with reasonable school sizes, allowing the use of school means with confidence intervals, the lower variance component sometimes associated with schools results in low values for generalizability coefficients and the dependability indices. Although limited data are available, it appears from the Connecticut assessment that tests with more extended responses need fewer items to achieve the same level of generalizability. Even with extended responses, at least two tasks are needed to achieve reasonable levels of generalizability. (Contains 43 tables and 5 references.) (SLD)
Descriptors: Elementary Secondary Education, Error of Measurement, Generalizability Theory, Performance Based Assessment, Reliability, State Programs, Testing Programs
Council of Chief State School Officers, One Massachusetts Avenue, Suite 700, Washington, DC 20001-1431 ($10). Tel: 202-408-5505; Fax: 202-408-8072. For full text: http://publications.ccsso.org
Publication Type: Reports - Research
Education Level: N/A
Sponsor: Office of Educational Research and Improvement (ED), Washington, DC.
Authoring Institution: Council of Chief State School Officers, Washington, DC.
Grant or Contract Numbers: N/A
Note: Produced by the State Collaborative on Assessment and Student Standards.