It is desirable for an educational assessment to be constructed of items that can differentiate different performance levels of test takers, and thus it is important to estimate accurately the item discrimination parameters in either classical test theory or item response theory. It is particularly challenging to do so when the sample sizes are…

This document describes a set of best practices for developing, implementing, and maintaining the critical process of scoring constructed-response tasks. These practices address both the use of human raters and automated scoring systems as part of the scoring process and cover the scoring of written, spoken, performance, or multimodal responses.…

This study uses a computerized formative assessment system that provides automated scoring and feedback to help students write scientific arguments in a climate change curriculum. We compared the effect of contextualized versus generic automated feedback on students' explanations of scientific claims and attributions of uncertainty to those…

In response to the calls for making key stakeholders' perspectives relevant in the test validation process, the study discussed in this report sought test-taker feedback as part of collecting validity evidence and supporting the ongoing field testing efforts of the new "TOEFL ITP"® Speaking section. Specifically, I aimed to investigate…

Researchers, theorists, and practitioners have expressed a renewed interest in the longitudinal dynamics of personality characteristics in adulthood, including organic life span trajectories and their amenability to volitional change. However, this research has apparently not yet expanded to include the Dark Triad (psychopathy, narcissism,…

Different variants of the selected-response (SR) item type have been developed for various reasons (i.e., simulating realistic situations, examining critical-thinking and/or problem-solving skills). Generally, the variants of SR item format are more complex than the traditional multiple-choice (MC) items, which may be more challenging to test…

This study examined how high school course-taking patterns (i.e., career and technical education [CTE] vs. academic vs. no concentration), personal characteristics embedded in a social cognitive theory framework (e.g., self-efficacy, academic expectations), and contextual variables (e.g., parental expectations, socioeconomic status [SES]) interact…

Noncredit community college programs provide an important route for workforce development. They offer affordable and accessible short-term training options for individuals seeking access to middle-skills jobs. Absent the burdens of accreditation standards, they can respond nimbly to local labor market needs. However, they can also be varied and…

Recognizing the appealing features of a tablet (e.g., an iPad), including size, mobility, touch screen display, and virtual keyboard, more educational professionals are moving away from larger laptop and desktop computers and turning to the iPad for their daily work, such as reading and writing. Following the results of a recent survey of…

Testing programs have explored the use of technology-enhanced items alongside traditional item types (e.g., multiple-choice and constructed-response items) as measurement evidence of latent constructs modeled with item response theory (IRT). In this report, we discuss considerations in applying IRT models to a particular type of adaptive testlet…

When constructed-response items are administered for a second time, it is necessary to evaluate whether the current Time B administration's raters have drifted from the scoring of the original administration at Time A. To study this, Time A papers are sampled and rescored by Time B scorers. Commonly the scores are compared using the proportion of…

The time is now to examine the nation's capacity to help guide students in gaining access to, paying for, and graduating from college. College promise programs have served as an excellent model. But because a uniform, national college promise model would not adequately serve the estimated 20 million students in postsecondary education, ETS and…

The United States continues to be a top destination for international students pursuing an advanced degree. Some information about the characteristics of international students applying to graduate programs in the United States is available, but little is known about how these characteristics are related to test taker performance on graduate…

The career gains of obtaining a graduate degree are well established, but those from lower socioeconomic status (SES) and underrepresented demographic backgrounds have persistently been disadvantaged in earning those degrees. We aim to contribute to research on enhancing access, diversity, and equity to graduate education by providing insights…

Researchers and practitioners in postsecondary and workplace settings recognize the value of noncognitive constructs in predicting academic and vocational success but also perceive that many students or employees are lacking in these areas. In turn, there is increased interest in interventions designed to enhance these constructs. We provide an…