The study investigates how learner experience with applications (apps) and how app types (native vs. non-native) influence performance in reading comprehension. Twenty-five high school students used different apps to read four texts. The students were asked to use the annotation feature of the apps to facilitate their reading. Ease of use, usefulness, and enjoyment were self-reported. Cognitive load and valence were inferred from pupil diameter and facial expressions. Results showed no significant association between the number of annotations and reading comprehension. However, comprehension was better when the apps were perceived as easier to use, more useful, and more enjoyable. Regression analyses revealed that students performed significantly better when using native apps. There were also significant interactions between cognitive load and app types on reading comprehension, and between valence and app types on reading comprehension. Based on the students' feedback, basic requirements for an ideal annotation tool are identified.