The level of awareness of the importance of mastering ESL among the local community is poor, particularly in rural areas. A study was conducted to gather information from English as a Second Language (ESL) learners pertaining to the impediments of English Language learning (ELL) by specifically focusing on the four language skills namely listening, speaking, reading and writing. The study involved 30 Intermediate Year 5 pupils from a rural primary school. The method involved the use of note books as the medium for the learners to channel various problems which they feel hinder the effectiveness of their ESL learning in the four language skills. The pupils were also asked to suggest ways they think these problems can be solved by themselves, their ESL teachers and their fellow ESL peers. The analysis revealed varying viewpoints on the contributing factors that trigger various problems in learning the four skills as well as measures for mutual language improvement among them. It is hoped that ESL educators, particularly in rural areas will place importance on the various impediments which their ESL learners face and strive to make their learners' ELL more effective and meaningful.