A new reform movement in Japanese tertiary education has emerged which is more economy centered, more market sensitive, and more influenced by a government shift towards decentralization. With the dramatic decline in the 18 year old population, a buyer's market has led to the introduction of student evaluation of teaching surveys (SETs) partly as a measure student satisfaction. This is not without debate, and this study seeks to understand the perceptions of 22 local and expatriate English language teachers who participated in interviews. They suggest that using SETs as the sole criterion for evaluating teachers is flawed, unsystematic, and does not lead to improvement. Participants suggest the need for a model of "creative evaluation" over the present "creation of confusion." Teachers are unaware of the purpose of the evaluation which is not explained and often are just expected to administer without any consultation or input into the questions. Evaluation should draw distinction between prescriptive, acontextual summative evaluation and collaborative approaches that show the richness and diversity while giving learners as well as faculty more voice.