ERIC Number: ED381617
Record Type: RIE
Publication Date: 1993-Apr-15
Reference Count: N/A
The B.A.R. Demonstration Project: A Comparative Evaluation Trial of Computer-Based, Multimedia Simulation Testing and "Hands-on," Actual Equipment Testing.
Maher, Thomas G.
A general evaluation design was developed to examine the effectiveness of a computer-based, multimedia simulation test on California smog check mechanics. The simulation test operated on an Apple Macintosh IIci, with a single touchscreen color monitor controlling a videodisc player; it had three parts: introduction-tutorial-help, data, and test. It was compared to a hands-on, actual vehicle test. The two tests were as equivalent as possible. The hands-on test was essentially a printout of the nine test items and specific questions from the simulation test. The evaluation gathered two kinds of data: test performance and subjective responses to the test experience. Currently certified smog check mechanics were recruited and paid to participate in the study. This sample was randomized between the two testing methods; 38 were tested in each group. A Rasch analysis was chosen for a reliability study of the test. The high item separation reliability indicated that, in general, persons responded consistently to the test according to their performance capabilities. The comparison of the performance between the simulation test group and the hands-on test group indicated the difference was not statistically significant. Data showed that the probability was very high that the differences between the group means were the result of chance. (Appendixes which comprise at least half of the document include the following: background and simulation test documents, mechanics survey, sample selection documents, raw data score sheets, and test proctors' observations/recommendations.) (YLB)
Publication Type: Reports - Evaluative; Tests/Questionnaires
Education Level: N/A
Sponsor: California State Dept. of Consumer Affairs, Sacramento.
Authoring Institution: N/A