This paper presents the author's response to Xiaoming Xi's article titled "How do we go about investigating test fairness?" In this response, the author focuses on test fairness and Toulmin's model of argument structure, Xi's proposal, and the challenges the proposal brings. Xi proposes an approach to investigating test fairness to guide practitioners and illustrates it with the TOEFL iBT. Xi re-presents the six types of categories or inferences as modified by Chapelle et al. (2008) and, Bachman (2005) and Kane's (2006) argument-based approach. The author discusses a few key points of Xi's proposal. In conclusion, Xi has provided an interesting proposal that is both thought-provoking and challenging to address. The author hopes to see discussions on test fairness and evaluations of argument structures in the future as one tries to understand what test fairness means and how best to conduct test fairness research.