Journalartikel

Jahr der Veröffentlichung: 2021

Seiten: 454-474

Zeitschrift: Applied Stochastic Models in Business and Industry

Bandnummer: 37

Heftnummer: 3

ISSN: 1524-1904

eISSN: 1526-4025

Open Access Status: Hybrid

DOI Link: https://doi.org/10.1002/asmb.2574

Verlag: Wiley

Abstract: 
We draw attention to a problem that is often overlooked or ignored by companies practicing hypothesis testing (A/B testing) in online environments. We show that conducting experiments on limited inventory that is shared between variants in the experiment can lead to high false-positive rates since the core assumption of independence between the groups is violated. We provide a detailed analysis of the problem in a simplified setting whose parameters are informed by realistic scenarios. The setting we consider is a two-dimensional (2D) random walk in a semiinfinite strip. It is rich enough to take a finite inventory into account, but is at the same time simple enough to allow for a closed form of the false-positive probability. We prove that high false-positive rates can occur, and develop tools that are suitable to help design adequate tests in follow-up work. Our results also show that high false-negative rates may occur. The proofs rely on a functional limit theorem for the 2D random walk in a semiinfinite strip.

Harvard-Zitierstil: Bohle, D., Marynych, A. and Meiners, M. (2021) A fundamental problem of hypothesis testing with finite inventory in e-commerce, Applied Stochastic Models in Business and Industry, 37(3), pp. 454-474. https://doi.org/10.1002/asmb.2574

APA-Zitierstil: Bohle, D., Marynych, A., & Meiners, M. (2021). A fundamental problem of hypothesis testing with finite inventory in e-commerce. Applied Stochastic Models in Business and Industry. 37(3), 454-474. https://doi.org/10.1002/asmb.2574