Neighborhood-level screening algorithms are increasingly being deployed to
inform policy decisions. We evaluate one such algorithm, CalEnviroScreen -
designed to promote environmental justice and used to guide hundreds of
millions of dollars in public funding annually - assessing its potential for
allocative harm. We observe the model to be sensitive to subjective model
decisions, with 16% of tracts potentially changing designation, as well as
financially consequential, estimating the effect of its positive designations
as a 104% (62-145%) increase in funding, equivalent to \$2.08 billion
(\$1.56-2.41 billion) over four years. We also observe allocative tradeoffs and
susceptibility to manipulation, raising ethical concerns. We recommend
incorporating sensitivity analyses to mitigate allocative harm and
accountability mechanisms to prevent misuse