NotesFAQContact Us
Search Tips
ERIC Number: ED512655
Record Type: Non-Journal
Publication Date: 2010-Oct
Pages: 58
Abstractor: As Provided
Reference Count: 43
Developing High-Quality Assessments that Align with Instructional Video Games. CRESST Report 774
Vendlinski, Terry P.; Delacruz, Girlie C.; Buschang, Rebecca E.; Chung, Gregory K. W. K.; Baker, Eva L.
National Center for Research on Evaluation, Standards, and Student Testing (CRESST)
The evaluation of educational interventions requires assessments that consistently (reliably) produce data from which accurate (valid) inferences about the test subjects can be made for some stated purpose. Despite codified definitions of all these terms, there remains vibrant debate about the assessment design process and how measures of technical quality should be determined and reported. More importantly, there seems to be little consistency in the process of developing assessment items that demonstrate consistently high technical quality and that are conceptually aligned with instruction. We present a process, rooted in the work of Baker (1974), in which a theory of learning drives the development of knowledge specifications that then drive the integrated development of assessment items and forms and instructional materials. In this case, the instruction is in the form of an educational video game about adding rational numbers. We discuss the development of the knowledge and item specifications, an initial version of the instructional game based on these specifications, and report the consistently high technical quality (internal consistency and test-retest reliability--Cronbach's alpha of 0.9 to 0.94). We discuss as well the ability of pretest measures to predict subsequent student performance (inferential validity--Spearman's rho of 0.69) in the game and on similarly developed assessment measures. Alignment between the instructional game and assessment items is also discussed. Appendices include: (1) Knowledge Specifications; and (2) Puppetman Screen Shots from the First Version Tested with Students. (Contains 7 tables and 5 footnotes.) [This work was supported under the Educational Research and Development Centers Program, PR/Award Number R305C080015.]
National Center for Research on Evaluation, Standards, and Student Testing (CRESST). 300 Charles E Young Drive N, GSE&IS Building 3rd Floor, Mailbox 951522, Los Angeles, CA 90095-1522. Tel: 310-206-1532; Fax: 310-825-3883; Web site:
Publication Type: Reports - Evaluative
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: National Center for Research on Evaluation, Standards, and Student Testing