The purpose of this study is to investigate how learners of Japanese as a second language (n=16) and Japanese native speakers (n=17) interpret a Japanese refusal gesture, the so-called Hand Fan, to observe how these interpretations are accompanied by similar manual gestures, and to see how participants perceive its comprehensibility. Results indicate that learners are significantly poorer than native speakers at interpreting this uniquely Japanese refusal gesture, although there was no significant difference between the two groups in their judgments of the difficulty to interpret the Hand Fan gesture. This suggests that the acquisition of allegedly simple conventional gestures may not be so easy for language learners either for reception or production.