NotesFAQContact Us
Search Tips
Back to results
ERIC Number: ED561143
Record Type: Non-Journal
Publication Date: 2014
Pages: 119
Abstractor: As Provided
Measuring Deeper Learning through Cognitively Demanding Test Items: Results from the Analysis of Six National and International Exams. Research Report
Yuan, Kun; Le, Vi-Nhuan
RAND Corporation
In 2010, the William and Flora Hewlett Foundation's Education Program has established the Deeper Learning Initiative, which focuses on students' development of deeper learning skills (i.e., the mastery of core academic content, critical-thinking, problem-solving, collaboration, communication, and "learn-how-to-learn" skills). Two test consortia are developing the next generation of tests to measure students' attainment of the Common Core State Standards. These tests are expected to assess deeper learning skills to a greater extent than existing large-scale tests. A RAND study rated the cognitive demand of mathematics and English language arts items on six nationally and internationally administered exams: Advanced Placement, International Baccalaureate, the National Assessment of Educational Progress, the Programme for International Student Assessment, the Progress in International Reading Literacy Study, and the Trends in International Mathematics and Science Study, using Norman Webb's Depth of Knowledge framework and the Partnership for Assessment of Readiness for College and Career's self-developed frameworks. It found that these tests were more cognitively demanding than previously studied state achievement tests in both subjects, on average. The test items' level of cognitive demand varied by subject and format. The six tests varied in their percentages of cognitively demanding items, with only two tests meeting both criteria proposed by a panel of education researchers for high-quality measures of deeper learning. Moreover, the tests' cognitive demand levels varied with test purpose and the characteristics of the targeted students. The findings establish a benchmark for comparing how well the new generation of tests performs in assessing deeper learning. Findings included: (1) Six Nationally and Internationally Administered Tests Had Greater Cognitive Demand Than Previously Studied State Tests; (2) The Cognitive Demand of Test Items Varied by Subject and Item Format; and (3) The Six Tests Varied in the Extent to Which They Assessed Students' Deeper Learning Skills. The following are appended: (1) Distributions of NAEP Items, by Grade and Specific Framework Dimension; (2) Distributions of TIMSS Items, by Content and Cognitive Domain; (3) Exemplary Test Items at Each DOK Level; (4) Exemplary Test Items at Each PARCC Level, by Subject and Dimension; (5) Distribution of Modified PARCC Ratings; and (6) Results for the Original PARCC Dimensions and Levels.
RAND Corporation. P.O. Box 2138, Santa Monica, CA 90407-2138. Tel: 877-584-8642; Tel: 310-451-7002; Fax: 412-802-4981; e-mail:; Web site:
Publication Type: Reports - Research
Education Level: Grade 4; Intermediate Grades; Elementary Education; Elementary Secondary Education
Audience: Policymakers; Researchers; Practitioners
Language: English
Sponsor: William and Flora Hewlett Foundation
Authoring Institution: RAND Education
Identifiers - Assessments and Surveys: National Assessment of Educational Progress; Program for International Student Assessment; Progress in International Reading Literacy Study; Trends in International Mathematics and Science Study