Purpose of the Study: Although considerable attention has been paid to the relationship between later-life depression and cognitive function, the relationship between a history of psychiatric problems and cognitive function is not very well documented. Few studies of relationships between childhood health, childhood disadvantage, and cognitive function in later life consider both childhood health and disadvantage, include measures for psychiatric history, or use nationally representative longitudinal data. Design and Methods: This study uses growth curve models to analyze the relationships between childhood health and disadvantage, psychiatric history, and cognitive function using 6 waves of the Health and Retirement Study, controlling for demographics, health behavior, and health status. Results: A history of psychiatric problems is associated with lower cognitive function and steeper declines in cognitive function with age. The influence of childhood health is mediated by later-life health status and behaviors. A combined history of childhood disadvantage and psychiatric problems more strongly affects cognitive function, but cognitive declines remain consistent with those associated with psychiatric history. These effects are partially mediated by later-life demographic, socioeconomic, or health characteristics. Implications: These findings demonstrate that cumulative disadvantage and a history of psychiatric problems shape later-life cognition and cognitive decline. This evidence can enhance public understanding of the trajectories of cognitive decline experienced by groups living with disadvantage and can enable policy makers and human services providers to better design and implement preventative interventions and support services for affected populations.