NotesFAQContact Us
Collection
Advanced
Search Tips
ERIC Number: ED546192
Record Type: Non-Journal
Publication Date: 2012
Pages: 192
Abstractor: As Provided
Reference Count: N/A
ISBN: 978-1-2675-9347-4
ISSN: N/A
Data Collection Design for Equivalent Groups Equating: Using a Matrix Stratification Framework for Mixed-Format Assessment
Mbella, Kinge Keka
ProQuest LLC, Ph.D. Dissertation, The University of North Carolina at Greensboro
Mixed-format assessments are increasingly being used in large scale standardized assessments to measure a continuum of skills ranging from basic recall to higher order thinking skills. These assessments are usually comprised of a combination of (a) multiple-choice items which can be efficiently scored, have stable psychometric properties, and measure a broader range of concepts; and (b) constructed-response items that measure higher order thinking skills, but are associated with lower psychometric qualities and higher cost of test administration and scoring. The combination of such item types in a single test form complicates the use of psychometric procedures, particularly test equating which is a vital component in standardized assessment. Currently there is very little research that examines the robustness of current equating methodologies for tests that employ a mixed format. The purpose of this dissertation was twofold. The first goal of this research was to present evidence on the use of a predictive stratification framework based on an already available covariate to create equivalent groups. The second goal was to present supporting evidence on an appropriate data collection designs for mixed-format test equating. AP data from an AP Chemistry test and an AP Spanish Language test were obtained, covering a three year period. Two categorical covariates were created based on average AP score and school size from previous years. A 5 X 5 crosstab stratified cluster sampling matrix was created from the two new categorical variables and used to evaluate the accuracy and precision of mixed-format observed-score equipercentile equating. Six research conditions were investigated using a re-sampling framework as follows: (a) two random stratified cluster groups equating designs, (b) two test form conditions, (c) four sampling rates, (d) two AP test subjects, (e) two sampling frame conditions, and (f) three equating designs. There were two major findings summarized from the 500 bootstrap replications in each design condition. Firsts, the random stratified cluster group equating design had the most conditions with total equating error less than .1 standard deviation unit of the raw score scale. Second, Model 1, in which the equating function was estimated using a smaller sample and the larger sampling frame, was more accurate than Model 2 where the equating function was based on two equivalent samples from the stratified matrix. An unanticipated but interesting finding was that equating estimates from AP Spanish was more accurate compared to those from AP Chemistry despite the fact that the dis-attenuated correlation coefficient between the multiple-choice and constructed-response section was higher (unity) in AP Chemistry than in AP Spanish. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]
ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com/en-US/products/dissertations/individuals.shtml
Publication Type: Dissertations/Theses - Doctoral Dissertations
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A