Background: This study investigated the consequential validity of the International English Language Testing System (IELTS) Academic exam, specifically focusing on washback upon learners' test preparation strategies and score gain, and the mediating factors influencing washback when learners in an EFL context are not enrolled in test preparation courses. Method: Two IELTS Tests were administered to 190 undergraduates at a Japanese university over a 1-year period. A survey instrument was used to collect data about test preparation strategies for both tests. Test scores were compared to assess score gain. Interviews were conducted with 19 participants to investigate the factors mediating washback. Results: Test results revealed a significant increase in speaking ability, with more significant increases in speaking and listening for participants who reported preparing more intensely for the test. Survey results showed that students focused significantly more on speaking and writing, and significantly less on reading, when preparing for the second test, and those who prepared most intensely also focused significantly more on listening. A qualitative analysis of the interview data revealed a complex array of factors related to learner perceptions and their access to resources, which are highly dependent on learners' sociocultural and educational context, and which shape washback to the learner. Conclusions: The IELTS Test created positive washback on learners' language ability and test preparation strategies, specifically regarding productive skills, which learners in the study context had neglected in their previous language study. However, a range of mediating factors must be addressed in order to ensure positive washback in EFL contexts and in the absence of instruction.