Which hypothesis test should I use?
A quick decision guide to the right statistical test for your question and data type. Match your situation to the test, then check its assumptions.
Choosing the wrong test is one of the most common analysis mistakes. This guide maps the common situations to the standard test. It is a starting point, always check the test's assumptions before trusting the result.
| Your question | Data | Test |
|---|---|---|
| Is one mean different from a target? | One group, continuous | One-sample t-test |
| Do two groups' means differ? | Two independent groups | Two-sample t-test |
| Did a paired measure change? | Before/after on same units | Paired t-test |
| Do three or more groups' means differ? | 3+ groups, continuous | ANOVA |
| Are two categorical variables related? | Counts in categories | Chi-square test |
| Are two continuous variables related? | Two continuous | Correlation / regression |
| Is a proportion different from a target? | One group, pass/fail | One-proportion z-test |
| Do two proportions differ? | Two groups, pass/fail | Two-proportion z-test |
↓ Download as CSV · open data, free to reuse (CC BY 4.0)
These assume reasonably normal data for the parametric tests. For small or non-normal samples, consider non-parametric equivalents (Mann-Whitney, Wilcoxon, Kruskal-Wallis).
How to cite it.
Conyso. (2026). Which hypothesis test should I use? [Data set]. Conyso. https://conyso.com/reference-hypothesis-test-selector.html
DOI: pending — Zenodo archival plannedBench runs the common tests deterministically, with the working shown, not a black box.
Match the test to the question.
Three questions get you most of the way: is your outcome continuous or categorical, how many groups are you comparing, and are the groups independent or paired? Answer those and the right test usually follows from this table.
Then check assumptions, normality, equal variances, independence, before trusting the p-value. The right test on data that violates its assumptions is still the wrong answer.
Choosing a test, answered.
How do I choose a statistical test?
Identify your data type (continuous or categorical), the number of groups, and whether they are paired or independent. Those three answers point to the standard test.
What if my data is not normal?
Use a non-parametric equivalent, such as Mann-Whitney instead of a two-sample t-test, or transform the data. Small samples especially warrant caution.
When do I use ANOVA instead of a t-test?
Use a t-test to compare two group means and ANOVA for three or more. Running multiple t-tests across many groups inflates the false-positive rate, which ANOVA avoids by testing all groups at once.
What is the difference between a paired and an unpaired t-test?
A paired t-test compares two measurements on the same units (before and after), removing person-to-person variation. An unpaired (two-sample) t-test compares two separate, independent groups.