The aim of this study was to provide adaptive assistance to improve the listening comprehension of eleventh grade students. This study developed a video-based language learning system for handheld devices, using three levels of caption filtering adapted to student needs. Elementary level captioning excluded 220 English sight words (see Section 1 for definition), but provided captions and Chinese translations for the remaining words. Intermediate level excluded 1000 high frequency English words, but provided captions for the remaining words, and 2200 high frequency English words were excluded at the high intermediate caption filtering level. The result was that the viewers were provided with captions for words that were likely to be unfamiliar to them. Participants in the experimental group were assigned bilingual caption modes according to their pre-test results, while those in the control group were assigned standard caption modes. Our results indicate that students in the experimental group preferred adaptive captions, enjoyed the exercises more, and gained greater intrinsic motivation compared to those in the control group. The results confirm that different students require different quantities of information to balance listening comprehension and indicate that the proposed adaptive caption filtering approach may be an effective way to improve the skills required for listening proficiency.