NotesFAQContact Us
Search Tips
ERIC Number: ED511949
Record Type: Non-Journal
Publication Date: 2010-Jul
Pages: 92
Abstractor: ERIC
Reference Count: 5
Design Document for 12th Grade NAEP Preparedness Research Judgmental Standard Setting Studies: Setting Standards on the National Assessment of Educational Progress in Reading and Mathematics for 12th Grade Preparedness
National Assessment Governing Board
The National Assessment Governing Board adopted a Program of Preparedness Research in March 2009. Several categories of research studies were recommended to produce results for reporting 12th grade preparedness for the 2009 grade 12 National Assessment of Educational Progress (NAEP) in reading and mathematics. The categories included content alignment studies, statistical relationship studies, judgmental standard-setting studies, and surveys. The Governing Board will conduct a series of judgmental standard-setting studies to produce preparedness reference points on the NAEP scale for entry into job training programs and for placement in college credit-bearing courses. The purpose of this design document is to describe the procedures for the conduct of judgmental standard setting on the 2009 NAEP for grade 12 reading and mathematics. Specifically, the standard-setting activity is described in relation to the reading and mathematics skills and knowledge needed to qualify (a) for placement in entry-level credit-bearing postsecondary courses or (b) for placement in training programs in each of five occupations to be determined by the Governing Board. The design document describes the process to be implemented and the types of staff required to do the work. A modified bookmark method will be used as the judgmental standard-setting methodology. The goal is to maximize the comparability across the judgmental studies within and across the postsecondary activities. It is likely that more than one vendor will be required to implement the complete set of studies planned, and the information in this design document will allow the Governing Board to contract with multiple vendors to conduct comparable standard-setting activities for higher education and multiple occupational training programs. The design document is organized into three major sections: (1) Standard-setting Process; (2) Information Processing; and (3) Validity Evidence. Appendices include: (1) Sample Agenda for Judgmental Standard Setting; (2) Definition of 12th Grade Student Preparedness; (3) Example of Impact Data Questionnaire; (4) Examples of Process Evaluation Questionnaires; and (5) List of Twenty Potential Occupations. (Contains 19 figures and 2 tables.)
National Assessment Governing Board. 800 North Capital Street NW Suite 825, Washington, DC 20002. Tel: 202-357-6938; Fax: 202-357-6945; Web site:
Publication Type: Reports - Descriptive; Tests/Questionnaires
Education Level: Grade 12; High Schools; Higher Education; Postsecondary Education; Secondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: National Assessment Governing Board
Identifiers - Assessments and Surveys: National Assessment of Educational Progress