NotesFAQContact Us
Search Tips
Peer reviewed Peer reviewed
Direct linkDirect link
ERIC Number: EJ1011669
Record Type: Journal
Publication Date: 2013-Jan
Pages: 28
Abstractor: As Provided
Reference Count: 63
ISSN: ISSN-1093-023X
Web-Based Portfolio Assessment: Validation of an Open Source Platform
Collins, Regina; Elliot, Norbert; Klobucar, Andrew; Deek, Fadi P.
Journal of Interactive Learning Research, v24 n1 p5-32 Jan 2013
Assessment of educational outcomes through purchased tests is commonplace in the evaluation of individual student ability and of educational programs. Focusing on the assessment of writing performance in a longitudinal study of first-time, full-time students (n = 598), this research describes the design, use, and assessment of an open-source scoring platform. Augmenting usability testing, the research design relies on a framework of inter-reader agreement, inter-reader reliability, and coefficients of determination. The open-source, web-based portfolio assessment system yielded rates of agreement, reliability, and determination superior to the traditional paper-based portfolio assessment method. In addition, the system appears to be ideally suited to assess EPortfolios created to showcase student ability in digital environments: agreement range = 70% to 85%; reliability range = "kappa" = 0.67 ("p" less than 0.01) to "kappa" = 0.85 ("p" less than 0.01); coefficient of determination = "R[superscript 2]" = 0.95, "F"(5, 34) = 118.59 ("p" less than 0.01). This novel and innovative application of an open source platform for outcomes assessment yields the foundation for a sound validity argument, the control of human error, and complete system transparency and flexibility. Future research directions point to the need for the design and assessment of an open-source system designed to capture complex constructs as they emerge in digital environments. (Contains 3 tables.)
Association for the Advancement of Computing in Education. P.O. Box 1545, Chesapeake, VA 23327. Tel: 757-366-5606; Fax: 703-997-8760; e-mail:; Web site:
Publication Type: Journal Articles; Reports - Research
Education Level: Higher Education; Postsecondary Education
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A