ERIC Number: ED437412
Record Type: RIE
Publication Date: 1999-Nov-6
Reference Count: N/A
New Indicators for Program Evaluation.
Hanes, John C.; Hail, Michael
Many program evaluations involve some type of statistical testing to verify that the program has succeeded in accomplishing initially established goals. In many cases, this takes the form of null hypothesis significance testing (NHST) with t-tests, analysis of variance, or some form of the general linear model. This paper contends that, at least for program evaluation, the focus of NHST and its recommended alternatives misses the target of what evaluators really need to know about a program's success: (1) how meaningful to the client the changes attributable to the program are; (2) how many participants actually achieved these changes; and (3) how practical the program was. An approach is recommended that borrows from approaches used in the medical sciences to accumulate replicative evidence from repeated applications of the same program or from programs of a similar nature. This allows a defensible descriptive analysis of program effectiveness and efficiency. Taking a more descriptive approach means examining sustainable clinical improvement that results from the program. Using sustainable clinical improvement and the indicators of practical significance produces answers and makes statistical inference welcome, when warranted, but not necessary. (Contains 5 figures and 54 references.) (SLD)
Publication Type: Reports - Descriptive; Speeches/Meeting Papers
Education Level: N/A
Authoring Institution: N/A
Note: Paper presented at the Annual Meeting of the American Evaluation Association (14th, Orlando, FL, November 2-6, 1999).