T-Test vs Z-Test vs ANOVA

The z-test: σ known

A z-test compares a sample mean to a known value (or two groups) when the population standard deviation σ is known. This is rare in real research, because you rarely know σ. The test statistic is z = (x̄ − μ0)/(σ/√n). When σ is unknown, use a t-test instead — the PanelRoster z-test calculator states this requirement explicitly.

The t-test: σ unknown

The t-test replaces the unknown σ with the sample standard deviation s and accounts for the extra uncertainty with heavier tails. The one-sample t-test compares a mean to a known value; the independent-samples t-test compares two groups; the paired t-test compares two related measurements by testing the differences. Welch's t-test (the default in the PanelRoster calculator) does not assume equal variances and is a safer choice than the pooled Student's t-test.

ANOVA: three or more groups

One-way ANOVA compares the means of three or more independent groups at once. It partitions the total variation into between-group and within-group components and forms the F statistic, F = MS_between/MS_within. ANOVA controls the overall error rate when you compare several groups — running many pairwise t-tests instead inflates the chance of a false positive.

For exactly two independent groups, a two-tailed independent-samples t-test and a one-way ANOVA give equivalent results: the F statistic equals the square of the t statistic, and the p-values match.

Assumptions to check

These tests assume independent observations and approximately normal data (or a large enough sample). Student's t-test and ANOVA also assume roughly equal variances across groups; Welch's t-test relaxes that. If normality is doubtful, non-parametric alternatives such as the Mann-Whitney U test or Kruskal-Wallis test compare medians/ranks without the normality assumption.