Recent findings suggest deficits in coherent motion sensitivity, an index of visual dorsal stream functioning, in children with poor mathematical skills or dyscalculia, a specific learning disability in mathematics. We extended these data using a longitudinal design to unravel whether visual dorsal stream functioning is able to "predict" individual differences in subsequent "specific" mathematical skills, i.e., single-digit subtraction and multiplication. We measured children's sensitivity to coherent motion in kindergarten (mean age: 5 years 8 months) and evaluated their subtraction and multiplication skills in third grade (mean age 8 years 3 months). Findings revealed an association between subtraction but not multiplication performance and coherent motion sensitivity. This association remained significant even when intellectual ability and reading ability were additionally controlled for. Subtractions are typically solved by means of quantity-based procedural strategies, which reliably recruit the intraparietal sulcus. Against the background of a neural overlap between the intraparietal sulcus and visual dorsal stream functioning, we hypothesize that low-level visuospatial mechanisms might set constraints on the development of quantity representations, which are used during calculation, particularly in subtraction.