Higher education is back in the spotlight, with post-2015 sustainable development goals emphasising equality of access. In this paper, we highlight the long distance still to travel to achieve the goal of equal access to higher education for all, with a focus on poorer countries which tend to have lower levels of enrolment in higher education. Analysing Demographic and Health Survey data from 35 low- and middle-income countries in sub-Saharan Africa and South Asia, we show wide wealth inequalities in particular, with few if any of the poorest gaining access to higher education in some countries. We further identify that wealth and gender inequalities interact and tend to be wider in countries where levels of higher education are higher. This implies that expansion in access to higher education may predominantly benefit the rich, unless measures are taken to tackle inequalities. We find that that the rates of increase necessary for the attainment of the equal access goal by 2030 are particularly high. They pose a particularly difficult challenge given the access inequalities present from primary and secondary education in a wide majority of countries in our analysis. We therefore suggest that any measures aimed at attaining the goal need to tackle inequalities in access within a system-wide approach, focusing on the level of education at which inequalities initially manifest, alongside higher education.