We have developed a new in vitro skin irritation test based on an open source reconstructed epidermis (OS-REp) with openly accessible protocols for tissue production and test performance. Due to structural, mechanistic and procedural similarity, a blinded catch-up validation study for skin irritation according to OECD Performance Standards (PS) was conducted in three laboratories to promote regulatory acceptance, with OS-REp models produced at a single production site only. While overall sensitivity and predictive capacity met the PS requirements, overall specificity was only 57%. A thorough analysis of the test results led to the assumption that some of the false-positive classifications could have been evoked by volatile skin-irritating chemicals tested in the same culture plate as the non-irritants falsely predicted as irritants. With GC/MS and biological approaches the cross-contamination effect was confirmed and the experimental set-up adapted accordingly. Retesting of the affected chemicals with the improved experimental set-up and otherwise identical protocol resulted in correct classifications as non-irritants. Taking these re-test results into account, 93% overall sensitivity, 70% specificity and 82% accuracy was achieved, which is in accordance with the OECD PS. A sufficient reliability of the method was indicated by a within-laboratory-reproducibility of 85–95% and a between-laboratory-reproducibility of 90%