ERIC Number: ED492869
Record Type: Non-Journal
Publication Date: 2006-Jul
Automated Assessment of Domain Knowledge with Online Knowledge Mapping. CSE Technical Report 692
Chung, Gregory K. W. K.; Baker, Eva L.; Brill, David G.; Sinha, Ravi; Saadat, Farzad; Bewley, William L.
National Center for Research on Evaluation, Standards, and Student Testing (CRESST)
A critical first step in developing training systems is gathering quality information about a trainee's competency in a skill or knowledge domain. Such information includes an estimate of what the trainee knows prior to training, how much has been learned from training, how well the trainee may perform in future task situations, and whether to recommend remediation to bolster the trainee's knowledge. This paper describes the design, development, testing, and application of a Web-based tool designed to assess a trainee's understanding of a content domain in a distributed learning environment. The tool, called the CRESST Human Performance Knowledge Mapping Tool (HPKMT), enables trainees to express their understanding of a content area by creating graphical, network representations of concepts and links that define the relationships of concepts. Knowledge mappers have been used for several years, almost always as an aid for organizing information in support of problem solving or in instructional applications. To use knowledge maps as assessments there must be a reliable scoring method and there must be evidence for the validity of scores produced by the method. Further, to be practical in a distributed learning environment, the scoring should be automated. The HPKMT provides automated, reliable, and valid scoring, and its functionality and scoring method have been built from a base of empirical research. We review and evaluate alternative knowledge mapping scoring methods and online mapping systems. We then describe the overall design approach, functionality, scoring method, usability testing, and authoring capabilities of the CRESST HPKMT. The paper ends with descriptions of applications of the HPKMT to military training, limitations of the system, and next steps. (Contains 2 tables and 3 figures.)
Descriptors: Distance Education, Skill Analysis, Knowledge Level, Prior Learning, Internet, Concept Mapping, Scoring, Military Training, Test Validity, Test Construction, Evaluation Methods, Computer Software Evaluation, Computer Uses in Education, Programming, Computer Interfaces, Computer Software, Measures (Individuals)
National Center for Research on Evaluation, Standards, and Student Testing (CRESST). 300 Charles E Young Drive N, GSE&IS Building 3rd Floor, Mailbox 951522, Los Angeles, CA 90095-1522. Tel: 310-206-1532; Fax: 310-825-3883; Web site: http://www.cresst.org.
Publication Type: Reports - Descriptive
Education Level: N/A
Sponsor: Office of Naval Research, Washington, DC.
Authoring Institution: National Center for Research on Evaluation, Standards, and Student Testing, Los Angeles, CA.