This study investigated the role of item formats in the performance of 206 nonnative speakers of English on expressive skills (i.e., speaking and writing). Test scores were drawn from the field test of the "Pearson Test of English Academic" for Chinese, French, Hebrew, and Korean native speakers. Four item formats, including…

Research has shown that many educators do not understand the terminology or displays used in test score reports and that measurement error is a particularly challenging concept. We investigated graphical and verbal methods of representing measurement error associated with individual student scores. We created four alternative score reports, each…

This study provides initial evidence supporting intended score interpretations for the purpose of assessing opportunity to learn (OTL) via an online teacher log. MyiLOGS yields 5 scores related to instructional time, content, and quality. Based on data from 46 middle school classes, the evidence indicated that (a) MyiLOGS has high usability, (b)…

This study involved examination of the validity evidence of the Dynamic Indicators of Basic Early Literacy Skills-Next Edition (DIBELS Next) for a sample of 85 third-and fifth-grade students, in reference to the "simple view" of reading. Tests administered included DIBELS Next, Peabody Picture Vocabulary Test-IV (PPVT-IV), Group Reading…

We address valid testing for English language learners (ELLs)--students in the United States who are schooled in English while they are still acquiring English as a second language. Also, we address the need for procedures for systematically developing ELL testing accommodations--changes in tests intended to support ELLs to gain access to the…

Stanford University's Education Program for Gifted Youth (EPGY) conducted a randomized-treatment experiment during the 2006-2007 school year to test the efficacy, for Title I students, of the technological and individualized EPGY Kindergarten through Grade 5 Mathematics Course Sequence, modified for the Title I schools. Restricting attention…

We examined the performance of English language learners (ELLs) and non-ELLs on Grade 4 and Grade 5 mathematics content knowledge (CK) and academic language (AL) tests. CK and AL items had different semiotic loads (numbers of different types of semiotic features) and different semiotic structures (relative frequencies of different semiotic…

This study explores the value in using the difference between a student's predicted first-year grade point average (FYGPA), based on high school grade point average and SAT scores, and their observed FYGPA, namely, their FYGPA residual, as a tool in identifying those at risk for leaving an institution. Specifically, this study examined…

In this article, we highlight the need for a precisely defined construct in score-based validation and discuss the contribution of cognitive theories to accurately and comprehensively defining the construct. We propose a framework for integrating cognitively based theoretical and empirical evidence to specify and evaluate the construct. We apply…

The purpose of this study was to validate inferences about scores of one task designed to measure subject matter knowledge and three tasks designed to measure aspects of pedagogical content knowledge. Evidence for the validity of inferences was based on two expectations. First, if tasks were sensitive to expertise, we would find group differences.…

Measurement scholars have recently constructed validity arguments in support of a variety of educational assessments, including classroom observation instruments. In this article, we note that users must examine the robustness of validity arguments to variation in the implementation of these instruments. We illustrate how such an analysis might be…

Accountability in higher education has increased, with more institutions requiring standardized tests. These tests are high stakes for institutions, but low-stakes test for students, who seldom experience consequences for their performance. This study describes how one psychology department improved students' scores on the Psychology Area…

Prior research has shown that there is substantial variability in the degree to which the SAT and high school grade point average (HSGPA) predict 1st-year college performance at different institutions. This article demonstrates the usefulness of multilevel modeling as a tool to uncover institutional characteristics that are associated with this…

Despite the similarities that researchers note between the cognitive processes and knowledge involved in reading and writing, there are students who are much stronger readers than writers and those who are much stronger writers than readers. The addition of the writing section to the SAT provides an opportunity to examine whether certain groups of…

Educational program assessment studies often use data from low-stakes tests to provide evidence of program quality. The validity of scores from such tests, however, is potentially threatened by examinee noneffort. This study investigated the extent to which one type of noneffort--rapid-guessing behavior--distorted the results from three types of…