The primary objective of this article was to critically evaluate the routine use of confirmatory factor analysis (CFA) for representing an individual's performance across a battery of executive function tasks. A conceptual review and statistical reanalysis of N = 10 studies that used CFA methods of EF tasks was undertaken. Despite evidence of…

Higher-order factor analysis is a widely used approach for analyzing the structure of a multidimensional test. Whenever first-order factors are correlated researchers are tempted to apply a higher-order factor model. But is this reasonable? What do the higher-order factors measure? What is their meaning? Willoughby, Holochwost, Blanton, and Blair…

Since Miyake and his colleagues (2000) published their seminal paper on the use of confirmatory factor analysis (CFA) to parse executive function (EF), CFA methods have become ubiquitous in EF research. In their interesting and thoughtful Focus article, "Executive Function: Formative Versus Reflective Measurement," Willoughby and…

Research into executive functioning (EF) has indeed grown exponentially across the past few decades, but as the Willoughby et al. critique makes clear, there remain fundamental questions to be resolved. The crux of their argument is built upon an examination of the confirmatory factor analysis (CFA) approach to understanding executive processes.…

This author has had the privilege of knowing Professor Maydeu-Olivares for almost a decade and although their paths cross only occasionally, such instances were always enjoyable and enlightening. Edwards states that Maydeu-Olivares' target article for this issue, ("Goodness-of-Fit Assessment of Item Response Theory Models") provides…

The article provides an overview of goodness-of-fit assessment methods for item response theory (IRT) models. It is now possible to obtain accurate "p"-values of the overall fit of the model if bivariate information statistics are used. Several alternative approaches are described. As the validity of inferences drawn on the fitted model…

Lee Cronbach once expressed the view that all roads lead to construct validity. In looking to clarify the consensus definition of validity, and its place in assessment, Newton is also led to the troublesome idea of construct validity. To be sure, he addresses other validity issues, but in this commentary, I will restrict my attention to construct…

Mroch, Suh, Kane, & Ripkey (2009); Suh, Mroch, Kane, & Ripkey (2009); and Kane, Mroch, Suh, & Ripkey (2009) provided elucidating discussions on critical properties of linear equating methods under the nonequivalent groups with anchor test (NEAT) design. In this popular equating design, two test forms are administered to different groups of…

The paper by Drs. Rupp and Templin provides a much needed step toward the general application of diagnostic classification modeling (DCMs). The authors have provided a summary of many of the concepts that one must consider to properly apply a DCM (which ranges from model selection and estimation, to assessing the appropriateness of the model using…

Rupp and Templin (2008) do a good job at describing the ever expanding landscape of Diagnostic Classification Models (DCM). In many ways, their review article clearly points to some of the questions that need to be answered before DCMs can become part of the psychometric practitioners toolkit. Apart from the issues mentioned in this article that…

Nesselroade, Gerstorf, Hardy, and Ram developed a new and interesting way to enforce invariance at the second-order level in P-technique models, while allowing first-order structure to stray from invariance. We discuss our concerns with this approach under the headings of falsifiability, the nature of manifest variables included in models, and…

This article presents the authors' response which consists of three main parts. The first involves recapping the general thrust of their focus article. The second part consists of some general points that they hope will clarify issues raised by the commentators that were not made as clearly as they should have been in the focus article. The third…

"Diagnostic classification models" (DCM) are frequently promoted by psychometricians as important modelling alternatives for analyzing response data in situations where multivariate classifications of respondents are made on the basis of multiple postulated latent skills. In this review paper, a definitional boundary of the space of DCM is…

In this paper the author examines the role of item response theory (IRT), particularly multidimensional item response theory (MIRT) in test validation from a validity argument perspective. The author provides justification for several structural assumptions and interpretations, taking care to describe the role he believes they should play in any…

Ideally, the unit of analysis in psychology is the individual. However, many psychological methods do not cope well, either at the level of construct definition or at the level of measurement, with individuality in behavior. There is little leeway for constructs to be both idiosyncratically tailored to the individual, and still identified as…