ERIC Number: ED483398
Record Type: Non-Journal
Publication Date: 2004-May
Intuitive Test Theory. CSE Report 631
Braun, Henry I.; Mislevy, Robert J.
US Department of Education
Psychologist Andrea diSessa coined the term "phenomenological primitives", or p-prims, to talk about nonexperts' reasoning about physical situations. P-prims are primitive in the sense that they stand without significant explanatory substructure or explanation. Examples are "Heavy objects fall faster than light objects" and "Continuing force is needed for continuing motion." P-prims are based on experience. They are not a coherent system; they may even contradict one another. People assemble from them a model of sorts to reason about a given situation. Intuitive physics is wrong from a physicists point of view, but it works just fine, play fetch with your dog or push a couch across the room. It fails when you want to build a skyscraper or send a rocket to the moon. This paper considers p-prims that underlie reasoning about assessment, the basis of what one might call intuitive test theory. Examples are "A test measures what it says at the top of the page," and "Scores from any two tests can be made interchangeable, with a little equating magic." Testing p-prims underlie discussions of test theory in the classroom, in the news, and in policy-making. Again, intuitive test theory works reasonably well for everyday uses like Friday?s math quiz. It fails when an adaptive test is designed, or when change is measured in the proportion of students reading Above Basic from a matrix-sampled assessment such as NAEP. This report was administered by the U.S. Department of Education, Office of Educational Research and Improvement.
Descriptors: Test Theory, Testing, Evaluation Methods, Scores, Student Evaluation, Test Interpretation
National Center for Research on Evaluation, Standards, and Student Testing (CRESST), Graduate School of Education & Information Studies, University of California, Los Angeles, Los Angeles, CA 90095-1522. Tel: 310-206-1532.
Publication Type: Reports - Research
Education Level: N/A
Audience: Practitioners; Policymakers
Sponsor: Institute of Education Sciences (ED), Washington, DC.
Authoring Institution: California Univ., Los Angeles. Center for the Study of Evaluation.