The purpose of the study was to measure students' reported test-taking effort and the relationship between reported effort and performance on the Trends in International Mathematics and Science Study (TIMSS) Advanced mathematics test. This was done in three countries participating in TIMSS Advanced 2008 (Sweden, Norway, and Slovenia), and the…

The Standards for Educational and Psychological Testing indicate that multiple sources of validity evidence should be used to support the interpretation of test scores. In the past decade, examinee response processes, as a source of validity evidence, have received increased attention. However, there have been relatively few methodological studies…

The "Standards for Educational and Psychological Testing" indicate that test instructions, and by extension item objectives, presented to examinees should be sufficiently clear and detailed to help ensure that they respond as developers intend them to respond (Standard 3.20; AERA, APA, & NCME, 1999). The present study investigates the use of…

The TOEFL[R] iBT has increased the length of each reading passage to better approximate academic reading at North American universities, resulting in a reduction in the number of passages on the reading section of the test. One of the concerns brought about by this change is whether the decrease in topic variety increases the likelihood that an…

The National Assessment Governing Board used a new method to set achievement level standards on the 2005 Grade 12 NAEP Math test. In this article, we summarize our independent evaluation of the process used to set these standards. The evaluation data included observations of the standard-setting meeting, observations of advisory committee meetings…

The objective of the present investigation was to compare the adequacy of two cognitive models for predicting examinee performance on a sample of algebra I and II items from the March 2005 administration of the SAT[TM]. The two models included one generated from verbal reports provided by 21 examinees as they solved the SAT[TM] items, and the…

Central to the standards-based assessment validation process is an examination of the alignment between state standards and test items. Several alignment analysis systems have emerged recently, but most rely on either traditional rating or matching techniques. Little, if any, analyses have been reported on the degree of consistency between the two…

This article examined two item-writing guidelines: the format of the item stem and homogeneity of the answer set. Answering the call of Haladyna, Downing, and Rodriguez (2002) for empirical tests of item writing guidelines and extending the work of Smith and Smith (1988) on differential use of item characteristics, a mock multiple-choice driver's…

Studied the effects of variations in screen size, resolution, and presentation delay on verbal and mathematics scores on a computerized test for 357 high school juniors. No significant differences were found for mathematics scores, but verbal scores were higher with the larger resolution display. (SLD)

Studied classroom assessment events in high school social studies classes using data from 12 assessment events in classes of a teacher researcher. Results for the 96 students involved support the conclusion that student perceptions of the task and self-efficacy, reported mental effort invested, goal orientation, and learning strategy use differed…

Examined the impact of using item parcels on ad hoc goodness-of-fit indexes in confirmatory factor analysis using the Arabic version of Sarason's Reactions to Tests scale. Data from 421 and 372 Arabic speaking students at an Israeli high school show that lower skewness and kurtosis and higher validity occur for parcels than for individual items.…

Examined convergent and discriminant validity of various self-efficacy measures across two studies, one involving 358 U.S. high school students and another involving 235 Korean female high school students. Across the studies the first-order confirmatory factor analyses provide support for both convergent validity of different self-efficacy…

Studied the effects of calculator use on performance on the Scholastic Assessment Test I through a questionnaire completed by 202,391 examinees in 1996 and 215,034 in 1997. Use of calculators was associated with higher performance, but the more able students were more likely to have calculators. Overall, the effects of calculators were found to be…

Studied the effects of teacher knowledge of rubrics on the achievement of high school students. Results favored the achievement of students whose teachers had received rubric training in three of eight comparisons, but the other five comparisons were not significant. Significant results were in algebra for selected response and constructed…

Studied the effects of test consequences, response formats, gender, and ethnicity on the mathematics and science sections of the Michigan High School Proficiency Test. Results for more than 11,000 students show that students taking constructed response and multiple choice formats performed better under high stakes conditions. Discusses gender and…