A Decision Theoretic Framework for Hypothesis and Significance Testing

Abstract

From its inception, statistical testing has been a controversial area. There are several philosophies of testing and inference, the most common among them being the so-called frequentist and Bayesian approaches. These approaches have often been viewed as at odds with one another. In this paper, we suggest that in many common testing scenarios this is not the case. We will approach testing from a decision theoretic standpoint, framing testing and inference as decisions to be made about a parameter. In doing so, we show that the commonly used methods of testing and inference answer different questions but can both provide valuable knowledge. We aim to help researchers move away from the viewpoint that one must be either a "frequentist" or a "Bayesian", as statisticians have often divided themselves in the past, and toward the recognition that both schools of thought can make relevant contributions to their research.