NotesFAQContact Us
Collection
Advanced
Search Tips
Back to results
ERIC Number: ED524787
Record Type: Non-Journal
Publication Date: 2010
Pages: 177
Abstractor: As Provided
ISBN: ISBN-978-1-1244-5008-7
ISSN: N/A
EISSN: N/A
Investigating the Justifiability of an Additional Test Use: An Application of Assessment Use Argument to an English as a Foreign Language Test
Wang, Huan
ProQuest LLC, Ph.D. Dissertation, University of California, Los Angeles
Multiple uses of the same assessment may present challenges for both the design and use of an assessment. Little advice, however, has been given to assessment developers as to how to understand the phenomena of multiple assessment use and meet the challenges these present. Particularly problematic is the case in which an assessment is used for purposes for which it was not intended or developed. Current frameworks for assessment design and assessment use justification, such as the Messickian conceptualization of a unified validity, an interpretive argument, and the Evidence-Centered Design, provide insight into such issues, but have varying degrees of limitation in guiding the description and evaluation of multiple uses of a given assessment and in relating these uses to assessment design and administration. The present study aims to assist test developers of a college-level English as a Foreign Language (EFL) proficiency test in identifying the most critical area(s) in the current test design and administration for supporting an added use of the testing program. To achieve this goal, the study compared the original and added assessment uses and linked the observed differences to desired modified or additional conditions for justifying the added use. The comparison and linking were guided by the assessment use argument (AUA) framework with expansion proposed by the researcher. Based on the comparison and linking results, five key modified warrants were identified for supporting the added use. Among them, the warrant of the equitability of decisions was identified as the potentially most questionable one. The warrant was elaborated into six sub-warrants, and the backing for each sub-warrant was related to the test design and administration. Two pieces of the backing were then chosen for further investigation. Specifically, information was collected to address issues on the construct of the paper test and the inter-rater inconsistency of the oral test. To facilitate the investigation, structural equation modeling and generalizability studies were conducted, as well as other procedures including two-group dependent t-tests and Spearman rank-order correlation studies. Based on the study results, it was concluded that the most critical area in the current test design and administration for supporting the added use is likely to he the observed substantial measurement errors of the paper test due to construct-irrelevant factors. Accordingly, test developers are recommended to focus most on identifying construct-irrelevant factors measured in the paper test and addressing them accordingly. Implications and limitations of the study were also discussed. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]
ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com/en-US/products/dissertations/individuals.shtml
Publication Type: Dissertations/Theses - Doctoral Dissertations
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A
Grant or Contract Numbers: N/A