To examine, by gender, the relationship between adverse events in childhood or adolescence and the increased risk of early mortality (before 80 years). The study sample included 941 participants of the English Longitudinal Study of Aging who died between 2007 and 2018. Data on socioeconomic status, infectious diseases, and parental stress in childhood or adolescence were collected at baseline (2006). Logistic regression models were adjusted by socioeconomic, behavioral and clinical variables. Having lived with only one parent (OR 3.79; p = 0.01), overprotection from the father (OR 1.12; p = 0.04) and having had an infectious disease in childhood or adolescence (OR 2.05; p = 0.01) were risk factors for mortality before the age of 80 in men. In women, overprotection from the father (OR 1.22; p < 0.01) was the only risk factor for mortality before the age of 80, whereas a low occupation of the head of the family (OR 0.58; p = 0.04) and greater care from the mother in childhood or adolescence (OR 0.86; p = 0.03) were protective factors. Independently of one’s current characteristics, having worse socioeconomic status and health in childhood or adolescence increased the risk of early mortality in men. Parental overprotection increased the risk of early mortality in both sexes, whereas maternal care favored longevity in women