Purpose: Making policy makers, researcher, education leaders, and assessment developers aware that what matters in education assessment is a wicked problem that cannot be easily solved following traditional approaches. Design/Approach/Methods: Starting from the questions that what matters in education assessment, this article presented such questions as a wicked problem because there is no consensus, not right or wrong answer, and certain solutions may lead to side effects on students and society. Therefore, a new approach of ecology should be involved, and different education outcomes or intended qualities of learners are presented in complex relationships. Findings: Deciding what matters in education assessment is a wicked question. It is not a tame or technology problem and can be resolved by any conventional approaches. What is pivotal now is to decipher what matters in education and then what should be measured and ultimately how to measure. The ecology and collaborate approach deliberated in this article could expedite such a process. Originality/Value: This article advocates paradigm change in understanding and resolving one of the most urgent problems in education. It provides an ecology explanation of the relationships that exist among the different education outcomes and students' qualities. By guiding through the dissecting of the problem step by step, this article has demonstrated a unique angle of understanding the wicked problem.