In the social sciences, latent traits often have a hierarchical structure, and data can be sampled from multiple levels. Both hierarchical latent traits and multilevel data can occur simultaneously. In this study, we developed a general class of item response theory models to accommodate both hierarchical latent traits and multilevel data. The…

The many-faceted Rasch (MFR) model has been used to evaluate the quality of ratings on constructed response assessments; however, it can also be used to evaluate the quality of judgments from panel-based standard setting procedures. The current study illustrates the use of the MFR model for examining the quality of ratings obtained from a standard…

Extensive research has examined the effectiveness of admissions tests for use in higher education. What has gone unexamined is the extent to which tests are similarly effective for predicting performance at both the master's and doctoral levels. This study empirically synthesizes previous studies to investigate whether or not the Graduate Record…

In an effort to standardize academic application procedures, the authors developed the Standardized Letters of Recommendation (SLR) to capture important cognitive and noncognitive qualities of graduate school candidates. The SLR, which consists of seven scales, is applied to an intern-selection scenario. Both professor ratings (n = 414) during the…

This study examined the factorial validity of scores on the newly developed Students' Evaluation of Teaching Effectiveness Rating Scale (SETERS) through a series of confirmatory and multilevel structures. Conventional confirmatory factor analyses using the total covariance and pooled within-covariance matrices from two midwestern universities…

The development of the Arizona Course/Instructor Evaluation Questionnaire is described. Related research is presented demonstrating the reliability and validity of the Arizona Course/Instructor Evaluation Questionnaire. (Author/JKS)

The development and implementation in a psychology department of a set of student rating-of-instruction items was discussed. The results of item descriptive statistics, correlational, and principal component analysis supported the construct validity of the items. (Author)

The development of the Student Opinion Survey is described in terms of construction of the initial item pool, sampling of items, formation of general and specific scales, and evaluation of psychometric properties. Internal consistency reliabilities, scale intercorrelations, and specific variances are reported. (Author/JKS)

College students viewed taped lectures which were systematically varied by lecturers's enthusiasm and amount of information presented. They then rated the lectures with a standard faculty evaluation form. Discriminant analysis was performed in order to derive optimal scoring methods for the form. (Author/JKS)

Dimensions of student attitude toward teacher behavior at the college level were investigated via factor analysis. Six factors were extracted and examined with a second sample and compared to dimensions discussed in the literature. (JKS)

The validity of a Hebrew version of the Illinois Course Evaluation Questionnaire was investigated in eighty courses in science and engineering at the Israel Institute of Technology. Factor analysis revealed a reasonable correspondence between empirical structure and that afforded by logic and intuition. (Author/JKS)

Consistent with the distinction between student ratings (ratings assigned) and class ratals (ratings received), various properties of the FACE Instrument scales were investigated. (RC)

The effects of different instructors and student Myers-Briggs Type Inventory personality characteristics on the accuracy of predicting 1041 introductory English composition grades from a single regression equation were investigated. Results suggested that grades were inflated in classes with high means and deflated in classes with low means.…

Using both the Behaviorally Anchored Rating Scales (BARS) and the Purdue University Scales, 727 undergraduates rated 32 instructors. The BARS had less halo effect, more leniency error, and lower interrater reliability. Both formats were valid. The two tests did not differ in rate discrimination or susceptibility to rating bias. (Author/GDC)