Mark J. Schervish, Teddy Seidenfeld and Joseph B. Kadane
It has long been known that the practice of testing all hypotheses at
the same level (such as 0.05), regardless of the distribution of the
data, is not consistent with Bayesian expected utility
maximization. According to deFinetti's ``Dutch Book'' argument,
procedures that are not consistent with expected utility maximization
are incoherent and they lead to gambles that are sure to lose no
matter what happens. In this paper, we propose a method to measure the
degree to which incoherent procedures are sure to lose, so that we can
distinguish slightly incoherent procedures from grossly incoherent
ones. We present a detailed analysis of testing a simple hypothesis
against a simple alternative as a case-study of how the method can work.
Keywords: Bookie; Brier score; Coherence; Escrow; Gambler; Hypothesis testing.