Language is one of the medium of expressing our ideas, feelings and emotions. And if we think about language in present world then English is one of the most used languages in the world and English is used as a second language in Bangladesh. English is introduced here at the primary level and its inclusion continues till the tertiary level of education. Most of the students of the primary schools in rural areas are weak in English language due to lack of skilled and trained teachers who are familiar to the modern methods and approaches of teaching and lack of materials for teaching in the classroom. Primary level English curriculum implementation is essential in Bangladesh to achieve the set English language competency in the rural areas. Students in the rural areas are performing poorly in English compared to their urban counterparts. Statistics showed that there was a gulf of difference between the facilities enjoyed by rural schools and urban schools. The study explores the challenges of teaching English language in rural areas in context of Bangladesh. This study investigated the factors affecting student's performance in English language in rural areas. Data were collected using interviews, classroom observation and questionnaire. Result of the study reveals that students were highly motivated to learn English for future expectations such as local and international communication, academic advancement and employment prospects. It also provide a scenario of English teaching system in rural areas of Bangladesh as well as the problems and prospects of English language in perspective of Bangladesh.