NotesFAQContact Us
Collection
Advanced
Search Tips
ERIC Number: ED546222
Record Type: Non-Journal
Publication Date: 2012
Pages: 141
Abstractor: As Provided
Reference Count: N/A
ISBN: 978-1-2676-0287-9
ISSN: N/A
A Simulation Study of the Situations in Which Reporting Subscores Can Add Value to Licensure Examinations
Feinberg, Richard A.
ProQuest LLC, Ph.D. Dissertation, University of Delaware
Subscores, also known as domain scores, diagnostic scores, or trait scores, can help determine test-takers' relative strengths and weaknesses and appropriately focus remediation. However, subscores often have poor psychometric properties, particularly reliability and distinctiveness (Folske, Gessaroli, & Swanson, 1999; Monaghan, 2006; Skorupski & Carvajal, 2010). The purpose of this study was to investigate the situations in which subscores in a licensure context can add value, thus be worth reporting. This research employed a simulation study looking at the particular characteristics of licensure examinations, such as widely divergent ability/difficulty distributions corresponding to the real life situation where a very high proportion of examinees pass. Conditions varied were number of items in the subtests, overlap between subtests, subscore inter-correlation, examinee ability level, and number of examinees. Subscores were calculated according to Haberman's (2008) methods and compared using his suggested evaluation criterion, the Proportion Reduction in Mean Square Error (PRMSE). Results indicated that licensure subscores are most likely to add value when they have high reliability and modest correlation with the total score, consistent with previous findings for subscores in general (Haberman, 2008; Sinharay, 2010). Since only a small percentage of subscores considered in this study added value (45/865 = 5%), results suggest that subscores are rarely worth reporting. Reporting recommendations were also provided to guide when to report subscores and how to properly build subtests to be worth reporting. These guidelines can help prevent the haphazard reporting of unreliable and misleading subscores that is occurring in the testing industry. [The dissertation citations contained here are published with the permission of ProQuest LLC. Further reproduction is prohibited without permission. Copies of dissertations may be obtained by Telephone (800) 1-800-521-0600. Web page: http://www.proquest.com/en-US/products/dissertations/individuals.shtml.]
ProQuest LLC. 789 East Eisenhower Parkway, P.O. Box 1346, Ann Arbor, MI 48106. Tel: 800-521-0600; Web site: http://www.proquest.com/en-US/products/dissertations/individuals.shtml
Publication Type: Dissertations/Theses - Doctoral Dissertations
Education Level: N/A
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: N/A