Multiple-choice questions are the basis of a significant portion of assessment in introductory economics courses. However, these questions, as found in course assessments, test banks, and textbooks, often fail to evaluate students' abilities to use and apply economic analysis. The authors conclude that multiple-choice questions can be used to measure some but not all elements of in-depth understanding of economics. The authors interpret in-depth understanding as ability to reason through logical steps when those steps and the relevant economic concepts are not explicitly stated. They present examples of multiple-choice questions that do and do not measure in-depth understanding.