Quality of Systematic Reviews of the Foods with Function Claims in Japan: Comparative Before- and After-Evaluation of Verification Reports by the Consumer Affairs Agency

Background: In Japan, a new type of foods with health claims, called Foods with Function Claims (FFC), was introduced in April 2015 in order to make more products available that are clearly labeled with certain health functions. Regarding substantiating product effectiveness, scientific evidence for the proposed function claims must be explained by systematic reviews (SRs), but the quality of SRs was not clear. The objectives of this review were to assess the quality of SRs based on the FFC registered on the Consumer Affairs Agency (CAA) website in Japan, and to determine whether the CAA’s verification report in 2016 was associated with improvement in the quality of SRs. Methods: We evaluated the reporting quality of each SR by the AMSTAR checklist on methodological quality. We searched the database from 1 April to 31 October 2015 as the before-SR and from 1 July 2017 to 31 January 2018 as the after-SR. Results: Among the 104 SRs reviewed, 96 final products were included: 51 (53.1%) were supplements, 42 (43.8%) were processed foods without supplements, and 3 (3.1%) were fresh foods. Of the 104 SRs, 92 (88.5%) were qualitative reviews (i.e., without meta-analysis) and 12 (11.5%) performed a meta-analysis. The average quality score of before-SRs and after-SRs was 6.2 ± 1.8 and 5.0 ± 1.9, respectively, a statistically significant decrease (p < 0.001). Conclusion: Overall, the methodology and reporting quality of after-SRs based on the FFC were poorer than those of before-SRs. In particular, there were very poor descriptions and/or implementations of study selection and data extraction, search strategy, evaluation methods for risk of bias, assessment of publication bias, and formulating conclusions based on methodological rigor and scientific quality of the included studies

A Cross-Sectional Study Based on Forty Systematic Reviews of Foods with Function Claims (FFC) in Japan: Quality Assessment Using AMSTAR 2

Background: The Foods with Function Claims (FFC) was introduced in Japan in April 2015 to make more products available that are labeled with health functions. The products’ functionality of function claims must be explained by scientific evidence presented in systematic reviews (SRs), but the quality of recent SRs is unclear. This study assessed the quality of SRs in the FFC registered on the Consumer Affairs Agency (CAA) website in Japan. Methods: We searched the database from 1 April to 31 October 2022. Confidence in the methodological quality of each SR was evaluated by the AMSTAR 2 checklist. Results: Forty SRs were randomly extracted on the basis of the eligibility criteria and recruitment procedures. Overall confidence was rated as “high” (N = 0, 0%), “moderate” (N = 0, 0%), “low” (N = 2, 5%), or “critically low” (N = 38, 95%). The mean AMSTAR 2 score was 51.1% (SD 12.1%; range 19–73%). Among the 40 SRs, the number of critical domain deficiencies was 4 in 7.5% of SRs, 3 in 52.5% of SRs, 2 in 35% of SRs, and 1 in 5% of SRs. Registering the review’s protocol and comprehensive search strategies were particularly common deficiencies. Additionally, the risk of bias (RoB) was insufficiently considered. Conclusion: Overall, the methodological quality of the SRs based on the FFC, introduced eight years earlier, was very poor. This was especially true in the interpretation and discussion of critical domains, which had many deficiencies in terms of protocol registration, a comprehensive literature search strategy, and accounting for the RoB