ERIC Number: ED335015
Record Type: RIE
Publication Date: 1991
Reference Count: N/A
The Validity of a Multiple-Choice, Paper and Pencil Instrument in Discriminating between Masters and Nonmasters of Instructional Design.
Stepp, Sidney Leland; Shrock, Sharon A.
While surveys have identified instructional design competencies, there has been virtually no systematic research of alternative means for assessing professional competence in this area. This paper reports on a study which investigated the question of whether a multiple-choice, paper and pencil test can validly discriminate between levels of professional competency in instructional design. The instructional design instrument was developed in three stages: (1) items were composed and revised until subject matter experts agreed to each item's logical validity; (2) trial testing and item analysis were done using groups of masters and nonmasters of instructional design to test empirically and eliminate non-discriminating items from the instrument; and (3) a phi coefficient was calculated to show a level of concurrent validity, the Tukey method of multiple range means testing was used to show significant differences between groups and subgroups, and multiple discriminant function analysis was used to identify additional items that did not discriminate between groups. A total of 257 subjects completed the instrument. A comparison of the mastery classification of the subject matter experts and the classification of instrument established the concurrent validity of the instrument, and it was concluded that this type of instrument can validly discriminate between masters and nonmasters of instructional design although further research is needed. A list of references used to identify the competencies and sample questions from the instrument are appended. (5 tables, 8 figures, and 37 references) (BBM)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Identifiers: Paper and Pencil Tests
Note: In: Proceedings of Selected Research Presentations at the Annual Convention of the Association for Educational Communications and Technology; see IR 015 132.