The aim of this study was to identify the effectiveness of a mobile application in vocabulary knowledge improvement of English as a Foreign Language (EFL) learners. This study adopted a quasi-experimental research design consisting of an experimental and a control group with 73 participants studying at two state universities in Turkey. During the application that lasted for 14 weeks, the experimental group tried to learn 40 collocations via CollocatApp, while control group used worksheets. The data were gathered through Collocation Achievement Test (CAT) developed by the researchers and Vocabulary Knowledge Scale (VKS) (Wesche & Paribakht, 1996). CAT was used for measuring receptive vocabulary knowledge of learners, and VKS was used for testing productive vocabulary knowledge. The findings showed that there was a significant difference between the experimental group and control group in terms of receptive vocabulary knowledge in the post-test. However, there was no difference between groups in retention tests. At the dimension of productive vocabulary knowledge, it was seen that there was no difference between the use of mobile applications and worksheets in terms of productive vocabulary knowledge. Thus, it was concluded that using mobile applications was an effective way of improving vocabulary knowledge receptively for only short-term memory.