ERIC Number: ED382667
Record Type: RIE
Publication Date: 1994-Apr
Reference Count: N/A
A Tree-Based Analysis of Items from an Assessment of Basic Mathematics Skills.
Sheehan, Kathleen; Mislevy, Robert J.
The operating characteristics of 114 mathematics pretest items from the Praxis I: Computer Based Test were analyzed in terms of item attributes and test developers' judgments of item difficulty. Item operating characteristics were defined as the difficulty, discrimination, and asymptote parameters of a three parameter logistic item response theory (IRT) model. Three types of item attributes were considered: (1) surface features (for example, whether or not the item stem contained an equation); (2) aspects of the solution process (for example, whether or not the solution required application of a standard formula); and (3) response type (free-response or multiple-choice). Because the attribute set included large numbers of categorical variables, an approach based on binary regression trees (Breiman, Friedman, Olshen, and Stone, 1984) was implemented. The results were quite impressive for asymptote parameters (85% of variance explained), somewhat less so for difficulty parameters (36% of variance explained) and fairly unimpressive for discrimination parameters (only 12% of variance explained). In addition, the tree-based approach was found to be particularly useful for identifying important interaction effects and for developing graphical summaries of the modeling results. Six tables and eight figures support the analyses. (Contains 11 references.) (Author/SLD)
Publication Type: Reports - Evaluative
Education Level: N/A
Authoring Institution: Educational Testing Service, Princeton, NJ.
Identifiers: Asymptotic Distributions; Free Response Test Items; Praxis I; Three Parameter Model; Tree Based Analysis