There are many reports of relations between age and cognitive variables and of relations between age and variables representing different aspects of brain structure and a few reports of relations between brain structure variables and cognitive variables. These findings have sometimes led to inferences that the age-related brain changes cause the age-related cognitive changes. Although this conclusion may well be true, it is widely recognized that simple correlations are not sufficient to warrant causal conclusions, and other types of correlational information, such as mediation and correlations between longitudinal brain changes and longitudinal cognitive changes, also have limitations with respect to causal inferences. These issues are discussed, and the existing results on relations of regional volume, white matter hyperintensities, and diffusion tensor imaging measures of white matter integrity to age and to measures of cognitive functioning are reviewed. It is concluded that at the current time the evidence that these aspects of brain structure are neuroanatomical substrates of age-related cognitive decline is weak. The final section contains several suggestions concerning measurement and methodology that may lead to stronger conclusions in the future. (Contains 3 tables and 5 figures.)