NotesFAQContact Us
Collection
Advanced
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1194310
Record Type: Journal
Publication Date: 2018
Pages: 17
Abstractor: As Provided
ISBN: N/A
ISSN: ISSN-0258-2236
EISSN: N/A
Available Date: N/A
Inter-Rater Agreement in Assigning Cognitive Demand to Life Sciences Examination Questions
Dempster, Edith R.; Kirby, Nicola F.
Perspectives in Education, v36 n1 p94-110 2018
Taxonomies of cognitive demand are frequently used to ensure that assessment tasks include questions ranging from low to high cognitive demand. This paper investigates inter-rater agreement among four evaluators on the cognitive demand of the South African National Senior Certificate Life Sciences examinations after training, practice and revision. The taxonomy used was based on the cognitive dimension of the Revised Bloom's Taxonomy, with analysis, evaluation and synthesis combined into one category. Descriptors from the Revised Bloom's taxonomy were slightly modified to suit Life Sciences. Inter-rater agreement was poor to fair, but pairwise percent agreement did not reach acceptable levels. Evaluators found it difficult to assign cognitive demand to examination items, and constantly referred to the descriptors. We question the usefulness of a taxonomy of cognitive demand when individuals differ in their interpretations of the levels of cognitive demand. The results indicate that standards of Life Sciences examination papers cannot reliably be assessed by evaluating cognitive demand using Bloom's Taxonomy.
University of the Free State Faculty of Education. P.O. Box 339, Bioemfontein 9300, South Africa. Tel: +27-51-401-2368; e-mail: PiE@ufs.ac.za; Web site: http://journals.ufs.ac.za/index.php/pie/index
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: South Africa
Grant or Contract Numbers: N/A
Author Affiliations: N/A