Reviews important milestones in the history of reliability, current issues related to reliability, and likely prospects for reliability from the perspective of what constitutes a replication of a measurement procedure. Pays special attention to the fixed/random aspects of facets that characterize replications. (SLD)

Examines testing practices in: (1) the past, in which the traditional paradigm left little room for cognitive psychology principles; (2) the present, in which testing research is enhanced by principles of cognitive psychology; and (3) the future, in which the potential of cognitive psychology should be fully realized through item design.…

Presents a method for combining multiple scale responses from job or task surveys based on a hierarchical rating scheme. Provides the rationale for placing the resulting ordinal information on an interval scale of measurement using the Rasch model. Also suggests a method for linking two or more surveys using the Rasch model and the BIGSTEPS…

Presents an approach to graphical differential item functioning (DIF) based on a sampling-theory approach to expected response functions. Applied the approach to a set of pretest items and compared results to traditional Mantel Haenszel DIF statistics. Discusses implications of the method as a complement to the approach of P. Pashley (1992). (SLD)

Describes four procedures previously developed for estimating conditional standard errors of measurement for scale scores and compares them in a simulation study. All four procedures appear viable. Recommends that test users select a procedure based on various factors such as the type of scale score of concern, test characteristics, assumptions…

Investigated the power and Type I error rate of the likelihood ratio goodness-of-fit statistic (LR) in detecting differential item functioning (DIF) under F. Samejima's (1969, 1972) graded response model. Monte Carlo study results show conditions under which the LR or Mantel Haenszel procedures have adequate power to detect DIF. (SLD)

Provides hypothesized explanations for local item dependence (LID) in a large-scale hands-on science performance assessment involving approximately 55,000 students each at grades 3, 5, and 8. Items that appear to elicit locally dependent responses require examinees to answer and explain their answers or to use given or generalized information to…

Investigated the effectiveness of H. Wainer's (1993) strategy for obtaining positively biased ability estimates when examinees can review and change answers on computerized adaptive tests. Results, based on simulation and testing data from 87 college students, show that the Wainer strategy sometimes yields inflated ability estimates and sometimes…

Studied effects of administration mode [computer adaptive test (CAT) versus self-adaptive test (SAT)], item-by-item answer feedback, and test anxiety on results from computerized vocabulary tests taken by 293 college students. CATs were more reliable than SATs, and administration time was less when feedback was provided. (SLD)

Proposes an algorithm for the assembly of multiple test forms in which the multiple-form problem is reduced to a series of computationally less intensive two-form problems. Illustrates how the method can be implemented using 0-1 linear programming and gives two examples. (SLD)

Describes a procedure for automated-test-forms assembly based on Classical Test Theory (CTT). The procedure uses stratified random-content sampling and test-form preequating to ensure both content and psychometric equivalence in generating virtually unlimited parallel forms. Extends the usefulness of CTT in automated test construction. (Author/SLD)

Compared results from computer-adaptive and self-adaptive tests under conditions in which item review was and was not permitted for 379 college students. Results suggest that, when given the opportunity, most examinees will change answers, but usually only to a small portion of items, resulting in some benefit to the test taker. (SLD)

Efficiency, precision, and concurrent validity of results from adaptive and fixed-item music listening tests were studied using: (1) 2,200 simulated examinees; (2) 204 live examinees; and (3) 172 live examinees. Results support the usefulness of adaptive tests for measuring skills that require aurally produced items. (SLD)

A method to measure speededness on tests is presented that reflects the tendency of examinees to guess rapidly on items as time expires. The method models response times with a two-state mixture model, as demonstrated with data from a computer-administered reasoning test taken by 7,218 examinees. (SLD)

Defines and demonstrates a framework for studying differential item functioning and differential test functioning for tests that are intended to be multidimensional. The procedure, which is illustrated with simulated data, is an extension of the unidimensional differential functioning of items and tests approach (N. Raju, W. van der Linden, and P.…