This article reports two studies intended to develop and assess conceptual models of how different factors mediate and moderate the annoyance reaction in school environments. In the first, a survey of 207 pupils was conducted where assumptions about mediators and moderators were formulated and tested. In the best model, general sensitivity and adaptation led to a higher degree of annoyance causing stress symptoms. In the second study, focus group interviews with 16 pupils were performed to set up a model of mediating and moderating factors from pupils' statements in the formation of annoyance. The objective was also to get their opinions about ways to improve the sound environment in school. The interviews indicated a serial arrangement in which stress symptoms and distraction mediated between chatter and disturbance. Thus, the two studies suggested different models for the prediction of the annoyance reaction. The pupils' views about how to improve the school sound environment are discussed in the framework of an empowerment model.