A rising number of Internet of Things (IoT) security and privacy threats have been documented over the last few years. However, IoT devices' domain designs are out-of-date and do not take into consideration the changing dangers associated with them. In this paper, we present COPSEC, a novel framework for evaluating whether IoT devices are compliant with security guidelines and privacy regulations. We extract metrics from existing guidelines and regulations and test them on a set of devices by performing hundreds of automated experiments. Our results indicate not only that these devices are not compliant with basic security guidelines, but also that their data collection operations may introduce privacy risks for the users that adopt them