ERIC Number: EJ701303
Record Type: Journal
Publication Date: 2004-Mar-1
Reference Count: N/A
Research: The Trouble with Research, Part 1
Bracey, Gerald W.
Phi Delta Kappan, v85 n7 p556 Mar 2004
Overall the past decade has not been kind to educational research. First, some "research" has been subordinated to and corrupted by ideology. Second, there has been substantial questioning of what educational research should be and a fear that the federal government is moving to a rigid orthodoxy in defining what counts as "science" or "research. Darrell Huff's 1954 publication, How to Lie with Statistics, is still in print and still worth reading. So is Bail Me Out!, by Gerald W. Bracey, the first third of which is called "Principles of Data Interpretation, or, How Not to Get Statistically Snookered." It's the flip side of Huff -- how to know when someone is lying to you with statistics. The techniques in How to Lie with Statistics have been increasingly practiced to influence education policy. Huff titled one of his chapters "The Gee Whiz Graph." In it, he first presents an appropriate graphic depiction of monthly national income increases over a year and says, "That is all very well if all you want to do is convey information. But suppose you wish to win an argument, shock a reader, move him into action. For that, this chart lacks pizzazz. Chop off the bottom." With the bottom half of the graph removed as suggested, the size of the increase is the same, but "what the hasty reader sees now is a national-income line that has climbed halfway up the paper in 12 months all because most of the chart isn't there anymore," Huff shows. He continues, "Now that you have practiced to deceive, why stop with truncating? You have a further trick that's worth a dozen of that. It will make your modest rise of 10% look livelier than one hundred percent is entitled to look. Simply change the proportion between the ordinate and the abscissa." Sadly, the U.S. Department of Education (ED) has been profligate in lying with statistics by means of Gee Whiz Graphs. In the January 2003 column an ED graph that showed increases in budgets for the Departments of Defense, Education, and Health and Human Services was reproduced. The graph made it appear that ED had received much more largesse than the other departments. But it hadn't. A situation in which subgroups show one trend and the aggregate shows a different trend is called Simpson's Paradox. It's common in statistics -- a Google search on it garnered 2,700 hits. It was explained it briefly in the Ninth Bracey Report (October 1999), and has also been explained in the February 2004 issue of the American School Board Journal. Education in America appears to be under attack, and one weapon of choice in the information age appears to be mis-information.
Descriptors: Politics of Education, Data Interpretation, Deception, Statistical Data, Federal Government, Educational Policy, Educational Research
Phi Delta Kappa International, Inc., 408 N. Union St., P.O. Box 789, Bloomington, IN 47402-0789. Web site: http://www.pdkintl.org.
Publication Type: Information Analyses; Journal Articles
Education Level: N/A
Authoring Institution: N/A
Identifiers - Laws, Policies, & Programs: No Child Left Behind Act 2001