In North American mathematics education, many practitioners highlight a disparity in achievement between Indigenous and non-Indigenous students, and claim that incorporating Indigenous perspectives in mathematics provides a more inclusive teaching approach. However, our analysis shows that there is a stream of North American practitioners who do not use anti-oppressive or decolonizing discourses, including those who claim to be motivated by social justice education. By avoiding or not emphasizing colonization, ongoing racism, and oppression in Indigenous mathematics education, these practitioners are perpetuating a false sense of the origins of inequality. Furthermore, the quest for Indigenous cultural connections in mathematics sometimes has consequences such as placing blame on Indigenous peoples for not being authorities on their cultures, perpetuating stereotypes, homogenizing Indigenous cultures while reducing their history and knowledge to superficial artifacts, and preserving a sense of the inferiority of Indigenous peoples when it comes to understanding and learning mathematics.