We address previous hypotheses about possible factors influencing the gender gap in attainment in physics. Specifically, previous studies claim that scaffolding may preferentially benefit female students, and we present some alternative conclusions surrounding this hypothesis. By taking both student attainment level and the degree of question scaffolding into account, we identify questions that exhibit real bias in favor of male students. We find that both multidimensional context and use of diagrams are common elements of such questions.