Although many innovations exploiting web technologies have been suggested in distance higher education, very little original research exists investigating the impact of web-based learning environments on distance students' learning processes and outcomes. To close this gap, four sets of data have been collected in a distance-learning course: behaviour and usage data derived from the log files of an online learning environment; responses from a competence pre- and posttest that measures the competence gain of online and offline students; technology assessment data from the online students capturing their perception of the learning environment; and formal assessment data acquired from a written examination test. By analysing these data, insight into the effectiveness of online learning material compared with custom-designed textbooks for self-controlled distance study can be gained. Distance educators and open or distance universities may take the analysis results as a basis for the decision whether to migrate from textbooks to online learning materials. A video abstract of this article can be viewed at https://youtu.be/sLF6PNmAYpY