To date, the historical entanglement of disability and happiness has not been considered an object worth of historical inquiry. Nor has the intersection of disability and emotions been used as a lens to examine the history of disability. Our paper aims at filling this academic void by analysing a wide range of philosophical, anthropological, pedagogical, popular and poetic texts dealing with the following question: "Who is unhappiest, the blind or the deaf?" On the basis of a comparative study of Belgian, Dutch, German, Swiss, Austrian, British and American primary source material, we argue that the transformation of western concepts of happiness during the nineteenth century was reflected in attitudes towards people with sensory disabilities. If at the turn of the eighteenth century happiness was considered a sensory experience, it very soon became dependent on the intervention of an educational expert. On the basis of our source material we draw the conclusion that happiness has been an overlooked factor in the construction and problematisation of disability in western societies. Analysis of the ways in which nineteenth-century authors have dealt with this question also shows how happiness became connected to new ways of wielding power, and in particular to the rise of the educational expert.