In this study, I investigated the reliability of the U.S. Naturalization Test's civics component by asking 414 individuals to take a mock U.S. citizenship test comprising civics test questions. Using an incomplete block design of six forms with 16 nonoverlapping items and four anchor items on each form (the anchors connected the six subsets of civics test items), I applied Rasch analysis to the data. The analysis estimated how difficult the items are, whether they are interchangeable, and how reliably they measure civics knowledge. In addition, I estimated how uniformly difficult the items are for noncitizens (N = 187) and citizens (N = 225) and how accurate the cutoff score is. Results demonstrated the items vary widely in difficulty and do not all reliably measure civics knowledge. Most items do not function differently for citizens and noncitizens. The cutoff is not as accurate as applied in the operational test. The data revealed that test scores contain construct-irrelevant variance that undermines the overall reliability and validity of the instrument. I discuss these results not only to better understand the civics test but also to recommend how United States Citizenship and Immigration Services could conduct a similar study with the goal of raising the reliability and validity of the test. (Contains 1 figure, 4 tables, and 10 footnotes.)