Many English learning websites have been developed worldwide, but little research has been conducted concerning the development of comprehensive evaluation criteria. The main purpose of this study is thus to construct a multi-dimensional set of criteria to help learners and teachers evaluate the quality of English learning websites. These evaluation guidelines are based on web usability, learning materials, functionality of assisting language learning, technology integration, and learner preferences. In order to achieve this goal, the researchers used a rigorous four-phase procedure and utilized both qualitative and quantitative research methods with university students and professors in Design-based Research to construct and refine the evaluation criteria in 2008-2009. Fifty-eight evaluation criteria were refined and finalized. In addition, to cope with the difficulties in applying the derived criteria and explaining what the results indicate, this study presents a practical application by evaluating an English learning website. (Contains 5 tables and 2 figures.)