ERIC Number: ED344936
Record Type: RIE
Publication Date: 1992-Apr
Reference Count: 0
Assessing Construct Validity Using Multidimensional Item Response Theory.
Ackerman, Terry A.
The concept of a user-specified validity sector is discussed. The idea of the validity sector combines the work of M. D. Reckase (1986) and R. Shealy and W. Stout (1991). Reckase developed a methodology to represent an item in a multidimensional latent space as a vector. Item vectors are computed using multidimensional item response theory item parameter estimates obtained from calibration programs such as NOHARM II (C. Fraser, 1983). The direction indicated by an item's vector indicates the composite of skills that the item is sensitive to measuring. Shealy and Stout developed a procedure (Simultaneous Item Bias) to detect item bias that encourages the user to identify (and condition on the scores from) only the most valid test items. T. A. Ackerman (1991) suggested that the most valid items for a given test could easily be identified from a plot of the item vectors. That is, the construct valid items should be measuring similar composites and thus lie within a definable sector. Items that lie outside this sector are assessing unintended-to-be-measured skills, and this could be considered construct invalid. This paper describes several uses of the validity sector, including how it can be used to construct tests, detect biased items, and define the raw score scale for a test. One table and 14 vector plots support the discussion. There is a 15-item list of references. (Author/SLD)
Publication Type: Reports - Evaluative; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Item Parameters; Validity Sector
Note: Paper presented at the Annual Meeting of the American Educational Research Association (San Francisco, CA, April 20-24, 1992).