Hypothesis Testing: An Intuitive Guide for Making Data Driven Decisions

The hypothesis of no difference is referred to as the null hypothesis. The p-value speaks to the credibility of the null hypothesis. A low p-value means the null hypothesis is less credible and unlikely to be true. If the null hypothesis is unlikely to be true, then it suggests our research hypothesis is true—specifically, there is a difference.

The null hypothesis is rejected if your data set is unlikely to have been produced by chance. The significance of the results is described by the confidence level that was defined by the test (as described by the acceptable error “alpha-level”).

With null hypothesis testing, all it takes is sufficient evidence (instead of definitive proof) that a 0 difference between means isn’t likely and you can operate as if at least some difference is true.

Even if the null hypothesis is rejected at a certain confidence level, no alternative hypothesis is proven thereby. The only conclusion you can draw is that some effect is going on. But you do not know its cause. If the experiment was designed properly, the only things that changed were the experimental conditions. So it is logical to attribute a c