This paper describes a controlled experiment concerning the use of a learning aid during an open and distance learning (ODL) course. The core issue of investigation is whether this instructional aid can support, guide, and scaffold the distant student in his/her study. For this purpose, a controlled experiment was conducted with the participation of 191 undergraduate students at the department of informatics at a university in Greece. The considered domain was two lessons concerning human-computer interaction (HCI), the first in usability engineering and the second in interface evaluation methodologies. A test session was also conducted to collect data on the assessment of the effectiveness of the proposed tool. Descriptive statistics as well as a variety of statistical methods have been applied to the collected data in order to test the research hypotheses. The results have shown a statistically significant difference in performance for the student group that used the tool. In order to validate these results, a validity evaluation was also considered and presented. Finally, concerns about the application of the tool in a broader context and further research on the area are also presented. (Contains 4 tables and 3 figures.)