Reduced speed in information processing is a well-documented phenomenon associated with advanced aging. Age-related deterioration in white matter integrity might play a role in age-related increase in reaction time (RT). However, the association between microstructural differences in particular white matter regions or tracts with RT is unclear. Decomposing RT into parts might be a better way to understand the relationship due to multiple processes involved in RT. In a lifespan sample of 90 healthy normotensive participants, this study examined the association between RT components derived from the Ratcliff diffusion model with age related difference in DTI indices of a wide variety of white matter tracts in both normal-appearing and whole white matter. The results revealed that advanced age was associated with lower drift rate, greater response conservativeness and longer non-decision time. Age-related reduction in FA and increase in MD was observed in most examined white matter tracts in both normal-appearing and whole white matter. Even in healthy normotensive adults, WMH burden could account for part of variance between age and DTI indices. Greater age-related difference in white matter integrity was observed in normotensive men than in normotensive women. Increased axial diffusivity of the superior corona radiata in normal appearing white matter was associated with longer non-decision time. However, there was no association between age-related differences in DTI indices of examined white matter tracts and both drift rate and response conservativeness in healthy normotensive participants