This study, based on total physical response and cognitive psychology, proposed a Kinesthetic English Learning System (KELS), which utilized Microsoft's Kinect technology to build kinesthetic interaction with life-related contexts in English. A subject test with 39 tenth-grade students was conducted following empirical research method in order to compare the effectiveness of KELS on both learning and motivation. In addition, we developed one questionnaire to study the perception learning style of the students involved in the experiment in order to determine whether students with different perception styles experienced any difference in learning while using the proposed KELS. Finally, we investigated students' acceptance of and motivation toward using KELS. The result showed that there was a significant difference between the control group and the experimental group regarding long-term retention. Moreover, the experimental group's learning was significantly affected by using kinesthetic interaction between peers to facilitate learning to speak and listen in English. The study also found that KELS was effective across perceptual learning styles. Finally, most participants in the experimental group agreed that KELS would effectively increase their motivation to learn English and expressed a strong intention to continue using the system.