Hygrothermal performance of building envelopes based on three different building materials (solid brick, high performance concrete, aerated autoclaved concrete) is assessed under climatic conditions of Prague and Atlantic City. Main objective of the paper is to evaluate the influence of cumulative damage, which was induced by means of natural weathering in 2012- 2015 period, on the long-term performance. The performance is assessed using various measures, namely time-of-wetness function, number of freeze/thaw cycles, and annual amount of energy transmitted through the envelope. The results show that thermal performance of the envelopes gets mostly better after weathering as the annual amount of energy is decreased by ~3.5% in average. On the other hand, time-of-wetness function and number of freeze/thaw cycles increase by ~24.1% and ~22.0%, respectively. Based on the results summary it can be concluded, that cumulative damage of materials has an indisputable influence on the hygrothermal performance of building envelopes which might be either negative or positive. A detailed computational assessment is therefore necessary, incorporating not only reference, but also weather-affected material properties