Abstract Objective: Little is currently known about the impact of congenital heart disease (CHD) on the organization of large‐scale brain networks in relation to neurobehavioral outcome. We investigated whether CHD might impact ADHD symptoms via changes in brain structural network topology in a cohort of adolescents with d‐transposition of the great arteries (d‐TGA) repaired with the arterial switch operation in early infancy and referent subjects. We also explored whether these effects might be modified by apolipoprotein E (APOE) genotype, as the APOE ε2 allele has been associated with worse neurodevelopmental outcomes after repair of d‐TGA in infancy. Methods: We applied graph analysis techniques to diffusion tensor imaging (DTI) data obtained from 47 d‐TGA adolescents and 29 healthy referents to construct measures of structural topology at the global and regional levels. We developed statistical mediation models revealing the respective contributions of d‐TGA, APOE genotype, and structural network topology on ADHD outcome as measured by the Connors ADHD/DSM‐IV Scales (CADS). Results: Changes in overall network connectivity, integration, and segregation mediated worse ADHD outcomes in d‐TGA patients compared to healthy referents; these changes were predominantly in the left and right intrahemispheric regional subnetworks. Exploratory analysis revealed that network topology also mediated detrimental effects of the APOE ε4 allele but improved neurobehavioral outcomes for the APOE ε2 allele. Conclusion: Our results suggest that disruption of organization of large‐scale networks may contribute to neurobehavioral dysfunction in adolescents with CHD and that this effect may interact with APOE genotype