ERIC Number: ED265188
Record Type: Non-Journal
Publication Date: 1985-Jun
Reference Count: N/A
Forecasting Device Effectiveness: Volume III. Analytic Assessment of Device Effectiveness Forecasting Technique. Final Report.
Rose, Andrew M.; And Others
This third of three volumes reports on analytic procedures conducted to address various aspects of the scalar properties of the Device Effectiveness Forecasting Technique (DEFT). DEFT, a series of microcomputer programs applied to data gathered from rating scales, is used to evaluate simulator devices used in U.S. Army weapons training. The analytic procedures included Monte Carlo simulations to assess the interpretation of DEFT data output; sensitivity of DEFT parameters; comparison of outputs; stability; and interrater agreement. Results indicated that it would be necessary to include assumptions regarding expected distributions of input variables in order to meaningfully interpret DEFT output. Also, the Monte Carlo analyses demonstrated the sensitivity of output scores to variations in inputs, and assessed the effects of various assumptions regarding measurement error on output scores. Interrater reliability was high for all three DEFT levels on ratings of three actual training devices, using several different raters. (GDC)
Descriptors: Adults, Computer Oriented Programs, Computer Simulation, Data Interpretation, Evaluation Methods, Instructional Material Evaluation, Interrater Reliability, Microcomputers, Military Training, Models, Monte Carlo Methods, Postsecondary Education, Prediction, Program Effectiveness, Program Evaluation, Rating Scales, Simulation, Skill Development, Statistical Analysis
Publication Type: Reports - Research
Education Level: N/A
Sponsor: Army Research Inst. for the Behavioral and Social Sciences, Alexandria, VA.
Authoring Institution: American Institutes for Research in the Behavioral Sciences, Washington, DC.