ERIC Number: EJ1049764
Record Type: Journal
Publication Date: 2015
Abstractor: As Provided
Reference Count: 43
Validating the Interpretations of PISA and TIMSS Tasks: A Rating Study
Rindermann, Heiner; Baumeister, Antonia E. E.
International Journal of Testing, v15 n1 p1-22 2015
Scholastic tests regard cognitive abilities to be domain-specific competences. However, high correlations between competences indicate either high task similarity or a dependence on common factors. The present rating study examined the validity of 12 Programme for International Student Assessment (PISA) and Third or Trends in International Mathematics and Science Study (TIMSS) tasks. Two tasks per competence (reading, mathematics, science, problem solving) from PISA and TIMSS were assessed by 34 teachers and 33 psychology students on 11 scales: difficulty, curriculum reference, knowledge versus thinking, reading competence, verbal ability, math competence, science competence, problem solving, reasoning, general knowledge, and intelligence. Intraclass correlation between two randomly chosen raters was r[subscript ic] =0.59. None of the tasks represented the intended target competence concisely. In five PISA tasks, competences other than those intended were seen as being more relevant. TIMSS tasks were seen as more curriculum-related and requiring more school knowledge than PISA tasks. For solving PISA tasks, thinking/reasoning ability and general intelligence were rated as being more important (d = 0.36). Only small differences were found between students' and teachers' ratings.
Descriptors: Test Validity, Test Interpretation, Competence, Reading Tests, Mathematics Tests, Science Tests, Problem Solving, Rating Scales, Correlation, Secondary School Teachers, Graduate Students, Interrater Reliability, Test Items, Foreign Countries
Routledge. Available from: Taylor & Francis, Ltd. 325 Chestnut Street Suite 800, Philadelphia, PA 19106. Tel: 800-354-1420; Fax: 215-625-2940; Web site: http://www.tandf.co.uk/journals
Publication Type: Journal Articles; Reports - Research
Education Level: Secondary Education; Higher Education; Postsecondary Education
Authoring Institution: N/A
Identifiers - Location: Germany
Identifiers - Assessments and Surveys: Program for International Student Assessment; Trends in International Mathematics and Science Study