2 research outputs found
A New Framework for Online Testing of Heterogeneous Treatment Effect
We propose a new framework for online testing of heterogeneous treatment
effects. The proposed test, named sequential score test (SST), is able to
control type I error under continuous monitoring and detect multi-dimensional
heterogeneous treatment effects. We provide an online p-value calculation for
SST, making it convenient for continuous monitoring, and extend our tests to
online multiple testing settings by controlling the false discovery rate. We
examine the empirical performance of the proposed tests and compare them with a
state-of-art online test, named mSPRT using simulations and a real data. The
results show that our proposed test controls type I error at any time, has
higher detection power and allows quick inference on online A/B testing.Comment: 8 pages, no figures. To be published on AAAI 2020 proceeding