Fermi problems, or order of magnitude estimates, are often used in introductory physics courses. In this paper I will show that first year students studying physics at university do not arrive with the skill set to solve these problems, and they have to be actively taught how to solve them. Once they have been shown how to solve Fermi problems, the above-average students in the class usually perform slightly better on Fermi problems in examinations than they do on more traditional physics problems. However, many students who are below average in overall grade find the Fermi problems even more challenging than the traditional problems. (Contains 5 figures.)