Turbulent flows past rough surfaces can create substantial energy losses in engineering equipment. During the last decades, developing accurate correlations to predict the thermal and hydrodynamic behavior of rough surfaces has proven to be a difficult challenge. In this work, we develop a convolutional neural network architecture to perform a direct image-to-image translation between the height map of a rough surface and its detailed local drag resistance and heat transfer rates. Various techniques are discussed to improve the computational efficiency of the machine learning architecture proposed, and even to reduce its time and space complexity. The main study is based on a new DNS database formed by 24 flow cases at a friction Reynolds number of Reτ = 180 obtained by applying a random shift to the Fourier spectrum of the grit-blasted surface scanned by Busse et al. (2015,). The results show that machine learning can accurately predict the global values of the drag resistance and heat fluxes across a rough surface. The local predictions for both momentum and heat transfer also show a considerable improvement upon increasing the dataset size. A detailed analysis of the global skin friction values and Stanton numbers predicted by deep learning further reveals that the results surpass the accuracy of traditional correlations by a substantial margin in the dataset analyzed.Energy TechnologyShip Hydromechanics and Structure