Applications extracting data from crowdsourcing platforms must deal with the
uncertainty of crowd answers in two different ways: first, by deriving
estimates of the correct value from the answers; second, by choosing crowd
questions whose answers are expected to minimize this uncertainty relative to
the overall data collection goal. Such problems are already challenging when we
assume that questions are unrelated and answers are independent, but they are
even more complicated when we assume that the unknown values follow hard
structural constraints (such as monotonicity).
In this vision paper, we examine how to formally address this issue with an
approach inspired by [Amsterdamer et al., 2013]. We describe a generalized
setting where we model constraints as linear inequalities, and use them to
guide the choice of crowd questions and the processing of answers. We present
the main challenges arising in this setting, and propose directions to solve
them.Comment: 8 pages, vision paper. To appear at UnCrowd 201
