The inclusion of the history and philosophy of science (HPS) in science teaching is widely accepted, but the actual state of implementation in schools is still poor. This article investigates possible reasons for this discrepancy. The demands science teachers associate with HPS-based teaching play an important role, since these determine teachers' decisions towards implementing its practices and ideas. We therefore investigate the perceptions of 8 HPS-experienced German middle school physics teachers within and beyond an HPS implementation project. Within focused interviews these teachers describe and evaluate the challenges of planning and conducting HPS-based physics lessons using collaboratively developed HPS teaching materials. The teachers highlight a number of obstacles to the implementation of HPS specific to this approach: finding and adapting HPS teaching material, knowing and using instructional design principles for HPS lessons, presenting history in a motivating way, dealing with students' problematic ideas about the history of science, conducting open-ended historical classroom investigations in the light of known historical outcomes, using historical investigations to teach modern science concepts, designing assessments to target HPS-specific learning outcomes, and justifying the HPS-approach against curriculum and colleagues. Teachers' perceived demands point out critical aspects of pedagogical content knowledge necessary for confident, comfortable and effective teaching of HPS-based science. They also indicate how HPS teacher education and the design of curricular materials can be improved to make implementing HPS into everyday teaching less demanding.