ERIC Number: ED476425
Record Type: RIE
Publication Date: 2003-Apr
Developing a Theory of Performance: A Two-Stage Structure for the Psychology of Standard Setting.
DeMauro, Gerald E.
An analysis was made of the cognitive processes that support the judgments made in standard setting activities. These processes were conceived as having two components: forming the domain needed to pass the test and identifying the criterion level of performance to pass the test. In fact, these processes are interactive, and were separated for the purposes of analysis and study. The judgments for the first component can be conceived of as a matrix in which the test questions provide information about information about whether or not their content is consistent with the test requirements. The second component can be conceived as a different type of matrix in which each test question is judged against a criterion value of difficulty. Analysis of judgments made by 38 panelists contributing to a mock standard setting suggests that the participating judges were consistently able to judge and use the judgments regarding the attributes of test items with respect to their conception of the required domain. They were also able consistently to judge the difficulty of the items against their criteria for passing the test. These findings suggest that this type of analysis may prove useful for future application in standard setting, and, certainly, for continued study of the process. (Author/SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Note: Paper presented at the Annual Meeting of the National Council on Measurement in Education (Chicago, IL, April 22-24, 2003).