There is no clear candidate tool for assessing the methodological quality of ex post facto studies in systematic reviews and meta-analyses yet. Our purpose is to thoroughly analyze the psychometric properties of the three most comprehensive assessment tools of this kind published up to 2010. We selected these tools from a previous systematic review, and we applied each one to assess the quality of 10 prospective studies, 10 retrospective studies with quasi-control group, and 10 cross-sectional studies. Inter-rater reliability for the first two aforementioned research designs is moderate only for one of the selected tools, and moderate to high for all of them for cross-sectional studies. Agreement between tools is low in general, although the inferred aspects show that the tools have a relative good conceptual overlapping in most of the domains. According to these results we recommend two tools for assessing cross-sectional studies, but we consider that the tools applicable to prospective studies or retrospective studies with quasi-control group require further testing. The 30 concrete aspects that we have inferred from the items of the three analyzed tools can be used as starting point to develop a new tool of this kin
