The aim of this study was to improve students' English as Foreign Language (EFL) writing and speaking performance with appropriate learning activity design supported by Virtual Pen ("VPen"), a multimedia web annotation system. Students' perceptions toward using "VPen" and learning activities, attitude toward using "VPen," actual "VPen" usage and their relationships were also investigated. After the experiment the students perceived that "VPen" was easy to use and useful during learning activities, and generally had a positive attitude toward using "VPen." Furthermore, the students believed that learning activities were "playful" and useful for improving writing and speaking performance. Further investigation revealed that the students' actual "VPen" usage had a significant correlation with speaking and writing performance and that students' speaking and writing performance significantly correlated with learning achievement. Based on our findings, we conclude that designed learning activities supported by the "VPen" system could facilitate students' writing and speaking performance and therefore improve their learning achievement.