PDF pending restoration
ERIC Number: ED167577
Record Type: RIE
Publication Date: 1979-Apr
Reference Count: 0
Some Applications of Generalizability Theory to the Dependability of Domain-Referenced Tests. ACT Technical Bulletin No. 32.
Brennan, Robert L.
Using the basic principles of generalizability theory, a psychometric model for domain-referenced interpretations is proposed, discussed, and illustrated. This approach, assuming an analysis of variance or linear model, is applicable to numerous data collection designs, including the traditional persons-crossed-with-items design, which is treated extensively here. It is shown that the appropriate error variance for domain-referenced interpretations is at least equal to and usually greater than the error variance for norm-referenced interpretations. Also, it is shown that two indices, error variance and signal/noise ratio, can be developed to reflect the dependability of a domain-referenced testing procedure. These indices are interpretable in several complementary ways, and they are easily compared with generalizability coefficients. In this paper, consideration is given not only to theoretical results but also to estimation procedures, illustrative examples, extensions to multiple facet designs, and recommendations for researchers and practitioners. Problems concerning loss functions and the setting of cutting scores are discussed. It is argued that differences between norm-referenced and domain-referenced tests are differences in score interpretation, whereas test development procedures define different domains. Venn diagrams are used to explain concepts. (Author/CTM)
Publication Type: Speeches/Meeting Papers; Reports - Research
Education Level: N/A
Authoring Institution: American Coll. Testing Program, Iowa City, IA. Research and Development Div.
Identifiers: Classical Test Theory; Domain Referenced Tests; Generalizability Theory; Signal Noise Ratios
Note: Paper presented at the Annual Meeting of the National Council on Measurement in Education (San Francisco, California, April, 1979) ; Not available in hard copy due to marginal legibility