Research on teaching and learning quantum physics (QP) frequently explores students' conceptual difficulties to identify common patterns in their reasoning. The abstractness of QP is often found to be at the origin of students' conceptual difficulties. Due to this abstract nature students resort to common sense reasoning or classical thinking when they make meaning of QP phenomena. In this literature review, the 'abstractness' is closely investigated and nuanced to uncover what reasons for the abstractness students experience. Four reasons for students' conceptual difficulties can be categorised under the abstract nature of QP. These reasons are that students struggle a) to relate the mathematical formalism of QP to experiences in the physical world; b) to interpret counterintuitive QP phenomena and concepts; c) to transit from a deterministic to a probabilistic worldview; and d) to understand the limitations of language to express quantum phenomena, concepts, and objects. Combining these four reasons allows us to better understand the origin of conceptual difficulties in QP and why these difficulties persist over time. The implications of these findings for research and teaching practice are discussed.