Capturing the temperature difference between sound and defective parts under ambient conditions is key for infrared thermography (IRT) on concrete bridges. This study explores the favorable time windows for concrete bridge deck inspections by IRT through field experiment and finite element model simulations. Based on the numerical simulations and experimental IRT results, the preferable thermal contrast to detect defects occurs during both daytime and nighttime. However, available time span during daytime is much shorter than that of nighttime due to interchange periods between cooling and heating cycles in the morning and in the evening. Furthermore, IRT is affected by sunlight during the daytime resulting in possible misdetections. Moreover, effects of clouds and radiative cooling are observed, and it is found that the clear sky is a preferable condition for IRT. Therefore, optimal conditions for IRT implementation on concrete bridge decks can be concluded that nighttime application under the clear sky condition. In addition, the effect of obstacles on a bridge surface such as gravel, wood chips that bring additional challenges to IRT are also evaluated experimentally