This study provides initial evidence supporting intended score interpretations for the purpose of assessing opportunity to learn (OTL) via an online teacher log. MyiLOGS yields 5 scores related to instructional time, content, and quality. Based on data from 46 middle school classes, the evidence indicated that (a) MyiLOGS has high usability, (b)…

This study validated the factor structure of a popular assessment of learner's cognitive test anxiety. Following recent findings in a study with Argentinean students' use of the Spanish version of the Cognitive Test Anxiety Scale (CTAS), this study tested the factor structure using data from 742 students who completed the original…

This study examines the development and implementation of a survey-based instrument assessing the effectiveness of a course redesign initiative focused on student centeredness at a large midwestern university in the United States. Given the scope of the reform initiative under investigation in this study, researchers developed an instrument called…

This study examines the potential of the tree diagram, a type of graphic organizer, as an assessment tool to measure students' knowledge structures in statistics education. Students' knowledge structures in statistics have not been sufficiently assessed in statistics, despite their importance. This article first presents the rationale and method…

Student-teacher interactions are dynamic relationships that change and evolve over the course of a school year. Measuring classroom quality through observations that focus on these interactions presents challenges when observations are conducted throughout the school year. Variability in observed scores could reflect true changes in the quality of…

In recent years, substantial investments have been made in reengineering systems of teacher evaluation. The new generation models of teacher evaluation typically adopt a standards-based view of teaching quality and include a value-added measure of growth in student learning. With more than a decade of experience and research, it is timely to…

English language learners (ELLs) constitute one of the fastest growing subpopulations of students in the United States. It is important to determine whether the assessments used by states in determining students' proficiencies are valid and fair for ELLs. This study focused on several standards-based assessments in mathematics and science…

This study compared the impact of state accountability examinations on social studies teachers' instructional practices. Data were obtained from a survey instrument given to a representative sample of Mississippi teachers who teach the same content tested on their state's high-stakes high school graduation examination and a representative sample…

Letters of reference are commonly used in acquiring a job in education. Despite serious issues of validity and reliability in writing and evaluating letters, there is a dearth of research that systematically examines the evaluation process and defines the constructs that define high quality letters. The current study used NVivo to examine 160…

We conducted generalizability studies to examine the extent to which ratings of language arts performance assignments, administered in a large, diverse, urban district to students in second through ninth grades, result in reliable and precise estimates of true student performance. The results highlight three important points when considering the…

As a preparer of students for the profession of teaching, a college, school, or department of education in higher education has an expectation that instructors teach competently. In higher education, the student survey of instruction is one of the most important among other indicators of teaching effectiveness. This article presents a validation…

Accepting that school based assessment may have the potential to bring additional reliability to the assessment outcomes of an educational system, this research uses Generalizability Theory to address the question "why school based assessment is not a universal feature of high stakes assessment systems"? Three major issues are identified: (a)…

Developed indicators of classroom practice using teacher assignments to monitor the influence of school reform initiatives on students' learning environments and to support the improvement of instructional practice. Results for classes of 32 third and seventh grade teachers show good reliability and consistency in ratings based on assignments.…

Examined the reliability and accuracy of item performance estimates from an Angoff standard setting application with 29 panelists on 1 year and 30 in the next year. Results provide evidence that item performance estimates were both reasonable and reliable. Discusses factors that might have influenced the results. (SLD)

Illustrates that there is a critical distinction between reliability and accuracy, explaining through a hypothetical example of shoe fitting that high test reliability does not guarantee good accuracy without consideration of percentile rank scoring, complex measurement models, and other technical detail. (SLD)