NotesFAQContact Us
Search Tips
Back to results
Peer reviewed Peer reviewed
PDF on ERIC Download full text
ERIC Number: ED578166
Record Type: Non-Journal
Publication Date: 2017-Dec
Pages: 73
Abstractor: ERIC
Scientific Evidence for the Validity of the New Mexico Kindergarten Observation Tool. REL 2018-281
Dahlke, Katie; Yang, Rui; Martínez, Carmen; Chavez, Suzette; Martin, Alejandra; Hawkinson, Laura; Shields, Joseph; Garland, Marshall; Carle, Jill
Regional Educational Laboratory Southwest
The New Mexico Public Education Department developed the Kindergarten Observation Tool (KOT) as a multidimensional observational measure of students' knowledge and skills at kindergarten entry. The primary purpose of the KOT is to inform instruction, so that kindergarten teachers can use the information about their students' knowledge and skills from the KOT to inform their curricular and pedagogical decisions. Stakeholders also are interested in using data from the KOT for other purposes, such as assessing student readiness for school statewide and identifying disparities in students' readiness for school across the state. This study examined the construct validity of the KOT to determine whether data from a field test supported using the KOT to measure six school readiness domains and, if not, what domain structure the data best supported. The study was conducted in response to the New Mexico Prekindergarten Research Alliance's and New Mexico Public Education Department's interest in evidence for the KOT's validity and reliability. The study team conducted exploratory and confirmatory factor analyses to identify the latent constructs that the KOT measured. The analyses identified the sets of items that were most related to one another and that could be used to develop domain scores. The study team also assessed reliability to confirm that items in the identified domains consistently measured the same construct. In addition, the study team applied other psychometric analysis methods to examine item functioning and differential item functioning across student subgroups. These analyses provided evidence of whether the KOT rating categories for each item were ordered correctly and whether there were any potential biases in how teachers rated student subgroups. Finally, the study team examined the proportion of the variation in the KOT's domain scores and item ratings at the classroom level to explore the extent to which domain scores and item ratings provide information about individual student abilities. The study identified valid and reliable approaches for scoring KOT item ratings, although not based on the developer's intended six-domain structure. Additional development and validation work is still needed to assess benchmarks for school readiness, determine whether particular items are biased for student subgroups, and examine the sources of classroom-level variations in scores.
Regional Educational Laboratory Southwest. Available from: Institute of Education Sciences. 555 New Jersey Avenue NW, Washington, DC 20208. Tel: 800-872-5327; Web site:
Publication Type: Reports - Research; Numerical/Quantitative Data
Education Level: Kindergarten
Audience: N/A
Language: English
Sponsor: N/A
Authoring Institution: National Center for Education Evaluation and Regional Assistance (ED); Regional Educational Laboratory Southwest (ED); Southwest Educational Development Laboratory
Identifiers - Location: New Mexico
Identifiers - Assessments and Surveys: Dynamic Indicators of Basic Early Literacy Skills (DIBELS)
IES Funded: Yes
Grant or Contract Numbers: EDIES12C0012