This article presents recent reform processes in Japanese higher education, concerning the tensions emerging within the system regarding "excellence" and "diversity". The article particularly focuses on how Japanese universities have reacted to the recent "competition" and "differentiation" policy promoted by the government, drawing on recent survey results conducted with academic managers at Japanese universities. It is interesting to examine the case of Japan, a historically diversified and differentiated national system, which has been changing rapidly with recent national "top-down" policy reforms, followed by more recent and new bottom-up institutional initiatives. The study shows that universities are trying to achieve "excellence," fulfilling different functions at the same time, aspiring to be excellent in teaching, research and social contribution without having institutional capacity to meet these expectations. Appropriate internal governance and external mediation mechanisms need to be created at the institutional level to manage "diversification" of the higher education system as a whole.