Optical clusters identified from red-sequence galaxies suffer from projection
effects, where interloper galaxies along the line-of-sight to a cluster are
mistaken as genuine members of the cluster. In the previous study (Sunayama et
al. 2020), we found that the projection effects cause the boost on the
amplitudes of clustering and lensing on large scale compared to the expected
amplitudes in the absence of any projection effects. These boosts are caused by
preferential selections of filamentary structure aligned to the line-of-sight
due to distance uncertainties in photometric surveys. We model the projection
effects with two simple assumptions and develop a novel method to quantify the
size of the boost using cluster-galaxy cross-correlation functions. We validate
our method using mock cluster catalogs built from cosmological N-body
simulations and find that we can obtain unbiased constraints on the boost
parameter with our model. We then apply our analysis on the SDSS redMaPPer
clusters and find that the size of the boost is roughly 20% for all the
richness bins except the cluster sample with the richness bin λ∈[30,40]. This is the first study to constrain the boost parameter independent
from cluster cosmology studies and provides a self-consistency test for the
projection effects.Comment: 13 pages, 11 figure