ERIC Number: EJ1019209
Record Type: Journal
Publication Date: 2014-Apr
Abstractor: As Provided
Reference Count: 12
A Practical Solution to Optimizing the Reliability of Teaching Observation Measures under Budget Constraints
Meyer, J. Patrick; Liu, Xiang; Mashburn, Andrew J.
Educational and Psychological Measurement, v74 n2 p280-291 Apr 2014
Researchers often use generalizability theory to estimate relative error variance and reliability in teaching observation measures. They also use it to plan future studies and design the best possible measurement procedures. However, designing the best possible measurement procedure comes at a cost, and researchers must stay within their budget when designing a study. In this study, we applied the LaGrange multiplier method to obtain facet sample size equations that minimize relative error variance (hence maximize reliability) under budget constraints. We did this for a crossed design and three nested designs that are more typical of data collection in teaching observation studies. Using an example budget and variance components similar to those found in practice, we demonstrate the use of these equations. We also show the way variance components from fully crossed designs can be combined to use our equations for a nested design.
Descriptors: Reliability, Classroom Observation Techniques, Generalizability Theory, Error of Measurement, Equations (Mathematics), Sample Size, Budgets, Costs, Research Design
SAGE Publications. 2455 Teller Road, Thousand Oaks, CA 91320. Tel: 800-818-7243; Tel: 805-499-9774; Fax: 800-583-2665; e-mail: email@example.com; Web site: http://sagepub.com
Publication Type: Journal Articles; Reports - Research
Education Level: N/A
Authoring Institution: N/A