At Golden College (California), student writing samples are holistically scored by pairs of judges on a six-point scale. Judges are allowed to use plus and minus figures, thus converting the integer scale to a decimal scale of evaluation. In 1991, 499 writing samples written as part of the placement testing process for students in the Coast…

Understanding the behavior of those evaluating a speech is important for a complete understanding of the public communication process. Ratings of public speaking were submitted to a Rasch analysis to determine whether objective measurement can create and maintain a standard of speech evaluation. Data used were 1,022 ratings of 168 speeches given…

A preliminary study examined the interrater reliability, factor structure, and convergent validity of the Haak Sentence Completion (in which the respondent is given a sentence stem and asked to provide the remainder of the sentence) using a three-point scoring system. The subjects were 100 boys ranging in age from 6 to 11 years who were randomly…

The purpose of this study was to evaluate the quality of experimental research in counseling and counselor education published from 1976 through 1984. The focus of the study was the methodology and reporting of the research rather than its relevance. Time was the independent variable of interest and three 3-year spans were chosen for examination:…

This study used generalizability theory and other statistical concepts to assess the application of the Angoff method to setting cutoff scores on two professional certification tests. A panel of ten judges gave pre- and post-feedback Angoff probability ratings of items of two forms of a professional certification test, and another panel of nine…

Cason and Cason's model of performance rating was used to determine the extent to which variation in reviewer standards affected the reliability and validity of the program review process used to select papers for inclusion in the annual program. Data analyzed were the overall recommendation for acceptance and ratings on seven quality criteria…

A more familiar and efficient method for estimating the parameters of Cason and Cason's model was examined. Using a two-step analysis based on linear regression, rather than the direct search interative procedure, gave about equally good results while providing a 33 to 1 computer processing time advantage, across 14 cohorts of junior medical…

A study explored (1) whether children demonstrated growth in writing ability through the experience of writing two or three times a week without direct teacher instruction; (2) what changes could be observed over time in the writing samples of second and fourth graders; (3) whether written language growth was a continuous progression of upward…

While diagnosis is generally considered a vital element in reading clinicians' expertise, research has revealed that even degreed, experienced reading clinicians display little personal consistency or agreement with one another when diagnosing simulated cases of reading difficulty. Three studies were conducted to determine if systematizing the…

Generalizability theory, which subsumes classical measurement theory as a special case, provides a general model for estimating the reliability of observational rating data by estimating the variance components of the measurement design. Research data from the "Heart Smart" health intervention program were analyzed as a heuristic tool.…

An experiment was designed that varied cutting score procedures, instructions, and types of judges in order to address the following questions concerning the Real Estate Licensing Examination: (1) Will the cutting score levels produced by groups of judges from differing backgrounds (academicians vs. practitioners vs. lawyers) using the same method…

Two native and one non-native (Japanese) instructors of English-as-a-foreign-language (EFL) corrected free compositions written by a Japanese college graduate with a degree in English. Analysis of the corrections revealed marked differences in type and number, with the non-native speaker most frequently indicating difficulty with articles, word…

The Vineland Adaptive Behavior Scale Classroom Edition and the American Association on Mental Deficiency Adaptive Behavior Scale School Edition were completed by teachers of 44 moderately and severely mentally retarded students; their parents (N=37) completed the Vineland Interview Edition-Survey Form. Comparison between the scales and respondents…

The Windward Rating Scale (WRS), a locally-developed teacher rating scale of student behavior, was evaluated for potential use as a screening measure. Pre-certification ratings of 720 learning disabled students and non-special education students in grades K-6 were analyzed. Psychometric properties and diagnostic efficiency of the WRS were…

This article studies the convergent-divergent validity and reliability estimates for clinical interview and self-report measures of ego identity. The findings suggest that the two measures may be: (1) assessing relatively distinct forms of ego identity; or (2) that the ego-identity construct as measured by the process and outcome dimensions needs…