This longitudinal study, spanning from 1995 through 2012, followed vulnerable youth from upper secondary school (T1) as they made the transition to their early twenties (T2), late twenties (T3) and mid-thirties (T4). We investigated their social network relationships in different phases of adult life, focusing mainly on factors that explain patterns of social adaptation. Internet-mediated relationships were also studied at T4. We found that attending special classes in upper secondary school explained the respondents' risk of being in a small isolated and bonding network in their early twenties. In the transition from early to late twenties, the relative impact of a changed life situation, which increases the potential of network formation, is a resilience factor. Logistic regression analyses, however, showed that in the transition from T3 to mid-thirties, having intellectual disabilities was a risk factor for social isolation. In particular, males with intellectual disabilities or psychosocial difficulties at this age were isolated in small networks and less likely to be on Facebook/Twitter. A shift of factors in the explanatory model of the transition from school to adult life reflects the relative impact of contextual factors distant and closer in time, shedding light on the principle of social ties to others, linked lives and cumulative disadvantages across the life course.