ERIC Number: ED453221
Record Type: RIE
Publication Date: 2001-Apr-13
Evaluating Validity and Reliability of Classroom Assessments Using Secondary Data.
Mushi, Selina L. P.
Analysis of secondary data was used as a way to inform the researcher about the trends in her assessment practices over a 4-year period. This was an important initial step in an effort to develop and integrate high-quality classroom assessment tasks and make sense of assessment information for decision making. Scores from 26 groups of graduate and undergraduate education students from 3 universities in the United States were analyzed. Course goals, objectives, and syllabuses were analyzed. Students' backgrounds and group combinations (age, gender, socioeconomic status) were taken into consideration in determining the consistency of specific assessment tasks in providing feedback to the instructor as researcher. The study results provided evidence of high content validity as well as high construct validity of time and nontimed assessment tasks. Concurrent validity among similar assessment tasks was evident. However, the predictive validity of assessment tasks (individual task to the final score) varied depending on whether the assessment task was nontimed (r=0.20 to r=0.41) or timed (up to 0.91). Timed assessment tasks were high predictors of the student's performance in the course (r=0.57 to r=0.01). Timed assessment tasks were more reliable (consistent) than nontimed tasks in providing assessment feedback across similar groups and contexts (contextual reliability). Scores from the nontimed assessment tasks fluctuated more from group to group (r=0.04 to r=0.68) than scores from timed tasks. Nontimed tasks probably tapped student skills and strategies that were not retrievable through timed examinations. The study highlights the importance of understanding, documenting, and evaluating assessment practices to better inform decision making at both classroom and program levels. (Contains 2 figures, 3 tables, and 17 references.) (SLD)
Publication Type: Reports - Research; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Note: Paper presented at the Annual Meeting of the American Educational Research Association (Seattle, WA, April 10-14, 2001).