When comparing digits of different physical sizes, numerical and physical size interact. For example, in a numerical comparison task, people are faster to compare two digits when their numerical size (the relevant dimension) and physical size (the irrelevant dimension) are congruent than when they are incongruent. Two main accounts have been put forward to explain this "size congruity effect". According to the shared representation account, both numerical and physical size are mapped onto a shared analog magnitude representation. In contrast, the shared decisions account assumes that numerical size and physical size are initially processed separately, but interact at the decision level. We implement the shared decisions account in a computational model with a dual route framework and show that this model can simulate the modulation of the size congruity effect by numerical and physical distance. Using other tasks than comparison, we show that the model can simulate novel findings that cannot be explained by the shared representation account. (Contains 8 figures.)