Individual differences in Cattell-Horn-Carroll (CHC) cognitive abilities are related to individual differences in math problem solving. However, it is less clear whether cognitive abilities are associated with math problem solving directly or indirectly via math component skills and whether these relations differ across grade levels. We used multigroup structural equation models to examine direct and indirect CHC-based cognitive ability relations with math problem solving across six grade-level groups using the Kaufman Assessment Battery for Children, Second Edition and the Kaufman Tests of Educational Achievement, Second Edition co-normed standardization sample data (N = 2,117). After testing factorial invariance of the cognitive constructs across grade levels, we assessed whether the main findings were similar across higher-order and bifactor models. In the higher-order model, the Crystallized Ability, Visual Processing, and Short-Term Memory constucts had direct and indirect relations with math problem solving, whereas the Learning Efficiency and Retrieval Fluency constructs had only indirect relations with math problem solving via math computation. The integrated cognitive ability and math achievement relations were generally consistent across the CHC models of intelligence. In the higher-order model, the g factor operated indirectly on math computation and math problem solving, whereas in the bifactor model, the first-order G factor had direct relations with math computation and math problem solving. In both models, g/G was the most consistent and largest cognitive predictor of math skills. Last, the relation of math computation with math problem solving increased as grade level increased. Theoretical implications for math development and considerations for school psychologists are discussed.