The popularity of smart learning has soared due to its flexibility, ubiquity, context-awareness, and adaptiveness. In particular, video-based m-learning has the biggest impact on the learning process. Its live and realistic features make learning interactive, easy, and fast. This article establishes the importance of video-based learning and m-learning in smart learning while discussing the basics of a smart learning environment and requirements. A framework and model for smart learning is presented. A streaming video adaptation model is also proposed for mobile devices. Based on the model, an interactive video-based smart learning system has been designed, which allows for the streaming video of live as well as prerecorded lecture sessions offering an interactive teaching-learning experience. The application supports both mobile devices and desktop computers. The model is practically implemented with a group of students and their feedback shows a high rate of acceptance of the system while a sizable percentage of them acknowledged that it improved their teaching-learning process significantly.