This paper argues that effective education in a knowledge society cannot be realized unless the issue of the policy of the language of instruction and assessment is resolved. This study discusses the relevance of the language policy currently in use for instruction and assessment of Ordinary Secondary Education in Tanzania. It also presents the research findings on the extent to which the language policy has passed a barrier towards students' performance and effective learning in post primary level. Data were collected through interviews, observation checklist and administering a test in English and Biology subjects to Form One students. Students who sat for the tests were purposively selected. The first group included 250 students who studied in English medium primary schools where English was a medium of instruction and the second group was 250 students who studied in Swahili medium primary schools where Swahili was a medium of instruction. Findings revealed that students who scored above 70 marks, the former English medium students had 78% in Biology and 87% in English test while the former Swahili medium students had 15% in Biology and 12% in English test. It was also revealed that 90% of the former English medium students were able to use computer technology in accessing learning materials written in English language compared to a few (0.5%) of the former Swahili medium students. The paper recommend that there is a need to remove the obstacle in Education and Training Policy and allow the establishment of government and private secondary schools which will be free to use English or Swahili as a medium of instruction and assessment. However Swahili and English subject should remain as compulsory subjects.