Research on English as a second language (ESL) students in higher education has traditionally focused on their academic writing, leaving larger issues of their college access and success unexplored. This article examines the challenges that first-generation immigrant and refugee ESL students face in accessing four-year college education through a qualitative interview study at a U.S. public university. Drawing on Bourdieu's cultural reproduction theory, we argue that what inhibits ESL students' access to and participation in four-year college education is not simply their limited English proficiency but also the structural constraints unique to this population, their limited financial resources, and the students' own tendency to self-eliminate. Based on our results, we call for a shift in higher education policy from one focusing narrowly on remediating ESL students' limited English proficiency to a more comprehensive set of policies that address the structural and economic, as well as linguistic, factors that together inhibit ESL students' college access and participation. (Contains 4 tables and 8 footnotes.)