Understanding an arithmetic operation implies, at minimum, knowing the direction of effects that the operation produces. However, many children and adults, even those who execute arithmetic procedures correctly, may lack this knowledge on some operations and types of numbers. To test this hypothesis, we presented preservice teachers (Study 1), middle school students (Study 2), and math and science majors at a selective university (Study 3) with a novel direction of effects task with fractions. On this task, participants were asked to predict without calculating whether the answer to an inequality would be larger or smaller than the larger fraction in the problem (e.g., "True or false: 31/56 * 17/42 > 31/56"). Both preservice teachers and middle school students correctly answered less often than chance on problems involving multiplication and division of fractions below 1, though they were consistently correct on all other types of problems. In contrast, the math and science students from the selective university were consistently correct on all items. Interestingly, the weak understanding of multiplication and division of fractions below 1 was present even among middle school students and preservice teachers who correctly executed the fraction arithmetic procedures and had highly accurate knowledge of the magnitudes of individual fractions, which ruled out several otherwise plausible interpretations of the findings. Theoretical and educational implications of the findings are discussed.