This article explores the potential for ethnographic observations to inform the analysis of test item performance. In 2010, a standardized, large-scale adult literacy assessment took place in Mongolia as part of the United Nations Educational, Scientific and Cultural Organization Literacy Assessment and Monitoring Programme (LAMP). In a novel form of interdisciplinary collaboration, an ethnographer worked closely with psychometric researchers to investigate the sources and explanations of differential item functioning and test item performance. The research involved detailed ethnographic observations of literacy assessment events. The results illustrate the potential for ethnography to provide insights into testing situations, group and test item performance, and to combine with psychometric analysis in cross-cultural assessment.