ERIC Number: ED211601
Record Type: RIE
Publication Date: 1979-May
Reference Count: 0
Qualitative Analysis of Test Item Attributes for Domain Referenced Content Validity Judgments. Studies in Measurement and Methodology, Work Unit 1: Design and Use of Tests.
Polin, Linda; Baker, Eva L.
This paper presents the interim results of a set of studies undertaken to develop a much needed methodology for establishing content validity in domain-referenced achievement tests. The study results are presented in the context of the larger issue of the improvement of test design. School teachers, administrators and graduate students were trained in using the Item Review Scale (IRS), and then served as item raters. The IRS provides structure for judgments of congruence between items and domain specifications. Raters were given a domain specification (elementary level geology) and eight related items to be rated for validity in terms of match. The items were systematically flawed along one or more of the dimensions of the IRS. Results demonstrated high inter-rater reliability on each dimension and the instrument as a whole. (Author/GK)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Sponsor: National Inst. of Education (DHEW), Washington, DC.
Authoring Institution: California Univ., Los Angeles. Center for the Study of Evaluation.
Identifiers: Domain Specifications; Inter Rater Reliability; Item Review Scale; Qualitative Analysis; Test Curriculum Overlap
Note: Paper presented at the Annual Meeting of the American Educational Research Association (San Francisco, CA, April, 1979).