In this article we describe our experiences using picture books to provide young children (five- to six-year-olds) with a learning environment where they can explore and extend preliminary notions of mathematics-related concepts, without being taught these concepts explicitly. We gained these experiences in the PICO-ma project, which aimed to generate more knowledge about the effect of picture books on young children's learning of mathematics. The project's goal is to investigate how picture books can contribute to the development of mathematical concepts in young children, and how the actions of the teacher can strengthen the characteristics of picture books that support learning. The reading sessions described in this article were not intended to be mathematics "lessons". Instead, the reading sessions were intended to tell the children a pleasant story and, at the same time, give them something to think about. Based on our research we provide reasons for using picture books to develop mathematical thinking, and include recommendations for practitioners interested in using picture books for mathematics learning. (Contains 7 figures.)