This study investigated whether early family-related environmental factors (FrF) can still affect the adults’ ability to recognize very brief facial expressions of emotion. The accuracy and RT to the decoding of microexpressions of anger, contempt, disgust, fear, happiness, and sadness was measured in a repeated-measures 6-AFC paradigm. Five experiments revealed that up to 13% of the variance in responses could be predicted by early FrF such as attachment orientation, number of siblings, and parental authority style. The effect sizes were modest to moderate. The adults’ accuracy for fear and anger was inversely correlated with the number of siblings, whereas the accuracy for contempt was positively correlated with an avoidant attachment or with recalled permissive and authoritative fathers. The RT for fear tended to be faster in adults with a fearful attachment, and faster for anger in those with permissive fathers. The findings point to long-lasting FrF effects on affective processing and suggest a role for FrF in the fine tuning of adults’ responses to signs of contextual hostility or threat when the time available for evaluation is severely restricted