Design activities allow students to create their own solutions, drawing upon a personal understanding of science principles and examples. We created the 'Houses in the Desert' project to engage middle school students in designing a passive solar house that will keep its owners comfortable in the desert climate. Students used their knowledge of thermodynamics to evaluate evidence and select between design alternatives. Classroom trials of the initial version (n = 139) of the project revealed four areas where it could be improved: (a) students ignored some relevant science content, (b) there was a lack of diversity in students' designs coupled with a tendency to fixate on initial design ideas, (c) opportunities for students to collaborate and share ideas were rare, and (d) there was limited opportunity for revisiting and revising ideas. We describe the revisions designed to address these challenges, as well as a new set of classroom trials (n = 140). A combination of comparative evaluations and observations of student work are used to evaluate the effectiveness of an instructional framework (Linn and Hsi 2000) for informing revisions to both the curriculum and the technology.