Pavement management systems aim to secure roadways functionality and vehicle passengers' safety by proposing strategies for pavement assessment and maintenance. However, transportation departments lack accurate, low-cost, and efficient methods for pavement assessment. Presented in this paper is a vision-based system for the detection of distressed pavement areas using low-cost technologies. Videos of pavement surface are recorded by a camera placed at the rear of a passenger vehicle, moving in a real-life urban network under normal traffic conditions. Collected data is processed by a developed algorithm that identifies video frames, including any type of pavement defect, using image entropy with a frame-based classification accuracy, precision, recall, and F1 score of 89.2%, 86.6%, 85.6%, and 86.1%, respectively. The proposed system can serve as the basis of any integrated pavement management system, saving significant amounts of time and cost for transportation departments