The current study applied the person-fit statistic, l[subscript z], to data from a Canadian provincial achievement test to explore the usefulness of conducting person-fit analysis on large-scale assessments. Item parameter estimates were compared before and after the misfitting student responses, as identified by l[subscript z], were removed. The…

Scholastic tests regard cognitive abilities to be domain-specific competences. However, high correlations between competences indicate either high task similarity or a dependence on common factors. The present rating study examined the validity of 12 Programme for International Student Assessment (PISA) and Third or Trends in International…

This study investigated whether the linguistic complexity of items leads to gender differential item functioning (DIF) on mathematics assessments. Two forms of a mathematics test were developed. The first form consisted of algebra items based on mathematical expressions, terms, and equations. In the second form, the same items were written as word…

The study examines the role that perceptions or impressions of learning environments and assessments play in students' performance on a large-scale standardized test. Hierarchical linear modeling (HLM) was used to test aspects of the Learning Errors and Formative Feedback model to determine how much variation in students' performance was…

Differential item functioning (DIF) analysis is important in terms of test fairness. While DIF analyses have mainly been conducted with manifest grouping variables, such as gender or race/ethnicity, it has been recently claimed that not only the grouping variables but also contextual variables pertaining to examinees should be considered in DIF…

The purpose of this study is to explore patterns in model-data fit related to subgroups of test takers from a large-scale writing assessment. Using data from the SAT, a calibration group was randomly selected to represent test takers who reported that English was their best language from the total population of test takers (N = 322,011). A…

After a test is developed, most content validation analyses shift from ascertaining domain definition to studying domain representation and relevance because the domain is assumed to be set once a test exists. We present an approach that allows for the examination of alternative domain structures based on extant test items. In our example based on…

As a construct definition, the current study develops a cognitive model describing the knowledge, skills, and abilities measured by critical reading test items on a high-stakes assessment used for selection decisions in the United States. Additionally, in order to establish generalizability of the construct meaning to other similarly structured…

This article reports the adaptation and analysis of Pintrich's Motivated Strategies for Learning Questionnaire (MSLQ) in Hong Kong. First, this study examined the psychometric qualities of the existing Chinese version of MSLQ (MSLQ-CV). Based on this examination, this study developed a revised Chinese version of MSLQ (MSLQ-RCV) for junior…

The main objective of the present study was to examine measurement invariance of the Reynolds Depression Adolescent Scale (RADS) (Reynolds, 1987) across gender and age in a representative sample of nonclinical adolescents. The sample was composed of 1,659 participants, 801 males (48.3%), with a mean age of 15.9 years (SD = 1.2). Confirmatory…

Very few empirically based studies have denied or confirmed the validity of holistic score interpretations and the validity of French-English writing scores comparisons. The present study addresses these important issues. Part I investigates if adjacent holistic scores represent different writing skills. Part II evaluates if variations exposed in…

This study uses PISA cycle 3 field trial data to investigate the relationships between gender differential item functioning (DIF) across countries and test languages for science items and their formats and the four other dimensions defined in PISA framework: focus, context, competency, and scientific knowledge. The data used were collected from 60…

Comparisons of human characteristics across different language groups and cultures become more important in today's educational assessment practices as evidenced by the increasing interest in international comparative studies. Within this context, the fairness of the results across different language and cultural groups draws the attention of…

The huge technological improvement in data processing and the globalization have increased the demand for and the supply of indices that quantify the consequences of a policy. However, there are certain cases in which quantification may be misleading in the sense that it gives the impression of an accurate measurement while in reality it is not.…