The majority of individuals with autism spectrum disorder live in low- and middle-income countries and receive little or no services from health or social care systems. The development and validation of autism spectrum disorder interventions has almost exclusively occurred in high-income countries, leaving many unanswered questions regarding what contextual factors would need to be considered to ensure the effectiveness of interventions in low- and middle-income countries. This study qualitatively explored contextual factors relevant to the adaptation of a caregiver-mediated early autism spectrum disorder intervention in a low-resource South African setting. We conducted four focus groups and four in-depth interviews with 28 caregivers of young children with autism spectrum disorder and used thematic analysis to identify key themes. Eight contextual factors including culture, language, location of treatment, cost of treatment, type of service provider, support, parenting practices, and stigma emerged as important. Caregivers reported a preference for an affordable, in-home, individualized early autism spectrum disorder intervention, where they have an active voice in shaping treatment goals. Distrust of community-based health workers and challenges associated with autism spectrum disorder-related stigma were identified. Recommendations that integrate caregiver preferences with the development of a low-cost and scalable caregiver-mediated early autism spectrum disorder intervention are included.