ERIC Number: ED440852
Record Type: RIE
Publication Date: 1999-Aug
Alignment of Science and Mathematics Standards and Assessments in Four States. Research Monograph No. 18.
Webb, Norman L.
Reviewers analyzed the alignment of assessments and standards in mathematics and science from four states at a four-day institute conducted June 29 through July 2, 1998. Six reviewers compared the match between assessment items and standards in mathematics and seven compared the match in science. Data from these analyses were processed and used to judge the degree of alignment on four criteria: categorical concurrence, depth-of-knowledge consistency, range-of-knowledge correspondence, and balance of representation. The analyses indicated that the standards of the four states varied in what content students were expected to know, the level of specificity at which expectations were expressed, and organization. Nearly all of the sixteen assessment instruments reviewed incorporated some constructed-response items. Only one mathematics assessment for grade 10 from one state consisted solely of multiple-choice items. The items in three science and two mathematics assessments analyzed from one state were evenly divided between multiple-choice and constructed-response items. Assessments from the other three states included from 80% to 90% multiple-choice items. Alignment between assessments and standards varied across grade levels, content areas, and states without any discernable pattern. Assessments and standards of three of the four states satisfied the categorical concurrence criterion. This criterion, the most common conception of alignment, required the assessment and standards to include the same content topics. Alignment was found to be the weakest on the depth-of-knowledge consistency and range-of-knowledge correspondence criteria. Generally, assessment items required a lower level of knowledge and did not span the full spectrum of knowledge as expressed in the standards. However, for the knowledge and skills identified in the standards and addressed by the assessments, generally the assessment items were evenly distributed. A major goal of this study was to develop a valid and reliable process for analyzing the alignment among standards and assessments. The process did produce credible results that distinguished among the different attributes of alignment and detected specific ways that alignment could be improved. Issues that did arise from an analysis of the process indicated that reviewers could benefit from more training at the beginning of the institute. Reviewers also needed more clarification of the four depth-of-knowledge levels and more explicit rules for assigning an assessment item to more than one statement of expectation. (Contains 15 references.) (CCM)
Descriptors: Academic Standards, Educational Assessment, Educational Change, Elementary Secondary Education, Evaluation Methods, Instructional Materials, Item Analysis, Mathematics Education, Science Education
Wisconsin Center for Education Research, 1025 W. Johnson Street, Madison, WI 53706 ($7.75). Tel: 608-265-9698.
Publication Type: Reports - Research
Education Level: N/A
Sponsor: National Science Foundation, Arlington, VA.
Authoring Institution: National Inst. for Science Education, Madison, WI.; Council of Chief State School Officers, Washington, DC.