PDF pending restoration
ERIC Number: ED395991
Record Type: RIE
Publication Date: 1995-Aug-14
Reference Count: N/A
Using Cluster Analysis To Solve the Problem of Standard Setting.
Sireci, Stephen G.
This paper presents and evaluates a new procedure to classify examinees into meaningful categories based on their responses to (performance on) items comprising a test instrument. The proposed procedure uses the data analytic technique of cluster analysis to partition the examinee population into homogeneous groupings. This procedure was applied to a high-stakes writing skills test. External self-reported grades in English composition classes were used to validate the cluster solutions. Two separate samples of 511 and 390 high school seniors from national equating samples for the Test of General Educational Development were analyzed to evaluate the stability of the obtained solutions. A five-cluster solution best separated the examinees according to their writing skills proficiency and led to a passing standard that was stable across the two samples. The benefits of using cluster analysis to set standards on tests include reduced subjectivity in comparison to methods based on expert ratings of items, applicability to performance-based assessments, and absence of dimensionality requirements. The limitations of the procedure include the need for external criterion data and the lack of research in this area. (Contains 10 tables and 13 references.) (Author/SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Dimensionality (Tests); Experts
Note: Paper presented at the Annual Meeting of the American Psychological Association (New York, NY, August 1995).