Negative numbers are among the first formalizations students encounter in their mathematics learning that clearly differ from out-of-school experiences. What has not sufficiently been addressed in previous research is the question of how students draw on their prior experiences when reasoning on negative numbers and how they infer from these experiences. This article presents results from an empirical study investigating sixth-grade students' reasoning and inferring from school-based and out-of-school experiences. In particular, it addresses the order relation, which deals with students' very first encounters with negative numbers. Here, students can reason in different ways, depending on the experiences they draw on. We study how students reason before a lesson series and how their reasoning is influenced through this lesson series where the number line and the context debts-and-assets are predominant. For grasping the reasoning's inferential and social nature and conducting in-depth analyses of two students' reasoning, we use an epistemological framework that is based on the philosophical theory of inferentialism. The results illustrate how the students infer their reasoning from out-of-school and from school-based experiences both before and after the lesson series. They reveal interesting phenomena not previously analyzed in the research on the order relation for integers.