With the globalization of world economy, English learners' writing ability has been attached less and less importance. As a result, many college students in China, especially the non-English majors, cannot express themselves effectively in written English. They make various kinds of mistakes, mostly grammar mistakes, such as writing sentence fragments, using dangling modifiers, and using punctuation marks inappropriately or incorrectly. Many factors are responsible for their weak writing ability, but teachers' neglecting grammar teaching and only teaching grammar knowledge are two critical factors. So, this paper is intended to call on teachers of English to reconsider grammar teaching for improving non-English majors' English writing ability. It first explores the role grammar plays in English learning and in English writing, then proposes a few practical and effective methods of instructing grammar, and finally concludes that to improve non-English majors' writing ability, we had better teach them the right grammatical concepts in the right way on the right occasion.