In the last decades, the notion that leadership comprises responsible leadership has gained support and the academic debate has shed some light on the antecedents, processes, and multi-level outcomes of responsible leadership. Being at the intersection of the leadership and sustainability discourses, responsible leadership has benefitted from the increasing interests that both fields of study have received. Nevertheless, the debate has left several questions around the nature and development of responsible leadership unanswered. Among these questions we reckon an understanding of "responsible" in the definition of "responsible leadership," the width of leaders' responsibility and the depth of their impact including the role of personal alongside formal leadership, and the distinction between "responsible" and "non responsible" leaders. The aim of this theoretical paper is to further the academic discussion on leadership in the context of sustainability and its integration in higher education settings. We review the literature and explore the academic debate while step-by-step building a description of responsible leadership that could form the basis for leadership programmes in higher education. Then, borrowing insights from pro-environmental psychology, we share a tripartite description of responsible leadership, which centres around identity, behaviour and responsiveness. As a final step, we share our experience in building an undergraduate programme based on this tripartite description of responsible leadership. Here, we illustrate how the leadership description can be visualised in a figure and used to develop an undergraduate Liberal Arts and Sciences curriculum centred on the UN Sustainable Development Goals.