There is a debate about the advantages and disadvantages of using social media in education. Drawing on interviews and surveys with students and teachers in three Swedish schools, this study finds that students as well as teachers find much of the students' social media use distractive to learning. We investigate this by means of an interpretative study of students' and teachers' experiences. We find that concerns relate to how social media use makes students less social, how weaker students are more likely to get distracted, how teachers lack strategies for tackling the problem and how the responsibility of the use is delegated to the students. We discuss how the distractive use of social media is made possible as a result of education policies requiring a higher degree of individual work, individual responsibility, and educational choices for students. Teachers and school leaders need to jointly reclaim the students and coping strategies for the distractive use are urgently needed.