NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ816384
Record Type: Journal
Publication Date: 2008-Oct
Pages: 10
Abstractor: As Provided
Reference Count: N/A
ISSN: ISSN-1382-4996
Development of a Content-Valid Standardized Orthopedic Assessment Tool (SOAT)
Lafave, Mark; Katz, Larry; Butterwick, Dale
Advances in Health Sciences Education, v13 n4 p397-406 Oct 2008
Content validation of an instrument that measures student performance in OSCE-type practical examinations is a critical step in a tool's overall validity and reliability [Hopkins (1998), "Educational and Psychological Measurement and Evaluation" (8th ed.). Toronto: Allyn & Bacon]. The purpose of the paper is to outline the process employed to create a content-valid Standardized Orthopedic Assessment Tool (SOAT). Orthopedic assessment skills are employed by athletic therapists, physiotherapists and physicians. All follow very similar diagnostic algorithms and that system was used to develop the original SOAT [Cyriax (1982). "Textbook of Orthopaedic Medicine", (vol. 1) Bailliere Tindall]. To validate the tool, the study followed procedures outlined by Violato, Salami, and Muiznieks (2002), "Journal of Manipulative Physiological Therapeutics", 25, 111-115, and Butterwick, Paskevich, Vallevand and Lafave (2006), "Journal of Allied Health": a modified Ebel procedure. An expert group of athletic therapists from across Canada were chosen to create the content valid tool. Representation from all accredited athletic therapy programs in Canada was sought. Experts participated in three stages of content validation: Stage one consisted of individuals grading tasks on difficulty (hard, medium, easy) and importance (essential, important, not important) for 8 body regions (cervical spine, lumbar spine, shoulder, elbow, wrist/hand/thumb, hip, knee and lower leg/foot/ankle) and three diagnoses for each body region (24 total). If 80% consensus was not achieved in the first stage, then in stage two, face to face discussion is meant to clarify positions and achieved consensus, if possible. The history component, the observation component, scanning exams, clearing joints above and below the lesion site and active range of motion, passive range of motion and isometric resisted testing all yielded 80% consensus in the first two stages of development. A third stage was added to this process because a new model of measurement was conceptualized during the second stage due to a lack of consensus on two orthopedic assessment categories: special testing and palpation. The new model consisted of a "clinical reasoning" metric that tied each section of an orthopedic assessment (history; observation; scanning and clearing; examination, special testing; palpation; conclusion) together and would permit the expert rater to evaluate the student performance based on the student's rationale for tests chosen rather than the traditionally rigid checklists. At least 80% consensus was achieved with the new "clinical reasoning" metric and the originally contentious special testing and palpation categories. The new SOAT that underwent content validation may be a hybrid between the traditional OSCE-type checklists and global scales that provide a more realistic and thus more valid depiction of orthopedic assessment clinical competence. Now that content validation has been established, the next steps include inter-rater reliability testing.
Springer. 233 Spring Street, New York, NY 10013. Tel: 800-777-4643; Tel: 212-460-1500; Fax: 212-348-4505; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Identifiers - Location: Canada