Research has shown that many educators do not understand the terminology or displays used in test score reports and that measurement error is a particularly challenging concept. We investigated graphical and verbal methods of representing measurement error associated with individual student scores. We created four alternative score reports, each…

The purpose of this study was to test the validity of the Teachers' Conceptions of Assessment Scale III-Abridged Version (CoA-IIIA; Brown, 2006), a measure created, validated, and applied outside of North America, in a sample of Canadian preservice teachers (n = 436). This work is important because although we have long known that…

Screening of students at risk for antisocial behaviors in school is an essential step in the implementation of evidence-based supports for academic, behavioral, and social domains at the first sign of concern. This study examined the measurement properties of a free-access systematic behavior screening tool: the Student Risk Screening Scale…

In the United States, government mandates for educational accountability have prompted an increase in testing in K-12 and higher education settings, resulting in a generation of millennial students who have undergone repeated assessment over the course of their educational careers. The cumulative effects of testing may range from increasingly…

The purpose of this study was to validate inferences about scores of one task designed to measure subject matter knowledge and three tasks designed to measure aspects of pedagogical content knowledge. Evidence for the validity of inferences was based on two expectations. First, if tasks were sensitive to expertise, we would find group differences.…

The task of validating a teacher assessment and improvement system is similar whether the system operates in the United States or in another country. Chile has a national teacher evaluation system (NTES) that is standards based, uses multiple instruments, and is intended to serve both formative and summative purposes. For the past 6 years the…

This article presents a study of ethnic Differential Item Functioning (DIF) for 4th-, 7th-, and 10th-grade reading items on a state criterion-referenced achievement test. The tests, administered 1997 to 2001, were composed of multiple-choice and constructed-response items. Item performance by focal groups (i.e., students from Asian/Pacific Island,…

Prior research has shown that there is substantial variability in the degree to which the SAT and high school grade point average (HSGPA) predict 1st-year college performance at different institutions. This article demonstrates the usefulness of multilevel modeling as a tool to uncover institutional characteristics that are associated with this…

An important part of test development is ensuring alignment between test forms and content standards. One common way of measuring alignment is the Webb (1997, 2007) alignment procedure. This article investigates (a) how well item writers understand components of the definition of Depth of Knowledge (DOK) from the Webb alignment procedure and (b)…

Both multiple-choice and constructed-response items have known advantages and disadvantages in measuring scientific inquiry. In this article we explore the function of explanation multiple-choice (EMC) items and examine how EMC items differ from traditional multiple-choice and constructed-response items in measuring scientific reasoning. A group…

This narrative synthesis reviews the psychometric properties of commercially and publicly available retell instruments used to assess the reading comprehension of students in grades K-12. Eleven instruments met selection criteria and were systematically coded for data related to the administration procedures, scoring procedures, and technical…

Students with disabilities participate in two major measurement systems. The Individuals with Disabilities Education Act emphasizes working within a Response to Intervention (RTI) framework to identify and monitor the progress of low-performing students. Persistent low-performing students also may be eligible for some form of an alternate…

To date, assessment validity research on non-native English speaking students in the United States has focused exclusively on those who are presently English language learners (ELLs). However, little, if any, research has been conducted on two other sizable groups of language minority students: (a) bilingual or multilingual students who were…

The Vanderbilt Assessment of Leadership in Education (VAL-ED) provides educators with a tool for principal evaluation based on principal, teacher, and supervisor reports of principals' learning-centered leadership. In this study, we conduct a known group analysis as part of a larger argument for the validity of the VAL-ED in US elementary and…

While feedback is a key factor for improving student learning, little is known about how students understand and experience feedback within the classroom. This study analysed 193 New Zealand primary and secondary students' survey responses alongside drawings of their understandings and experiences of feedback to examine how they experience,…