ERIC Number: ED458238
Record Type: Non-Journal
Publication Date: 2001-Apr
Reference Count: N/A
Model Criticism of Bayesian Networks with Latent Variables.
Williamson, David M.; Mislevy, Robert J.; Almond, Russell G.
This study investigated statistical methods for identifying errors in Bayesian networks (BN) with latent variables, as found in intelligent cognitive assessments. BN, commonly used in artificial intelligence systems, are promising mechanisms for scoring constructed-response examinations. The success of an intelligent assessment or tutoring system depends on the adequacy of the student model, representing the relationship between the unobservable cognitive variables of interest (thetas) and the observable features of task performance (x) with the probability model for x given theta expressed as a BN. The method for model fit analyses investigated in this study is appropriate for several uses in cognitive assessment. Data were generated for posited models to reflect the true BN model and several discrepancies from the true model. The study examined three indices: (1) Weaver's Surprise Index (Weaver, 1948); (2) Good's Logarithmic Score (Good, 1952); and (3) the Ranked Probability Score (Epstein, 1969). Simulation studies offer promise for the usefulness of the Ranked Probability Score and Weaver's Surprise Index as global measures and node measures to detect specific types of modeling errors in the latent structure of BNs. The introduction of this methodology and the emphasis on model criticism of BNs with latent variables provide a means of maximizing the accuracy and usefulness of BN models for a variety of applications. (Contains 4 tables, 9 figures, and 26 references.) (SLD)
Publication Type: Reports - Descriptive; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A