Practical Statistical Scenarios

This section shows how to pick the right statistical test and perform full workflows for common real-world situations:

• A/B testing (web experiments)
• Before/after tests (clinical & performance improvements)
• Survey & Likert-scale analysis
• Group comparisons (treatment groups, product versions)
• Check assumptions → run test → compute effect size → interpret
• Full “test selection workflow”

The focus is practical: what tests to run, how to run them, and how to interpret the results.

Scenario 1 — A/B Test (Conversion Rates)

Binary outcome → Proportion test

Question:
Is version B’s conversion rate higher than A’s?

Data:

• Group A: 200 conversions out of 2500 visits
• Group B: 260 conversions out of 2480 visits

Step 1: Extract counts

Step 2: Run the test

Step 3: Interpret

• p < 0.05 → conversion rates differ
• Check proportions:

Step 4: Effect size (Cohen’s h)

Step 5: Confidence intervals

Scenario 2 — Before/After Measurements (Paired Data)

Continuous → Paired t-test or Wilcoxon

Question:
Did a new process improve response time?

Data:

Step 1: Check normality of differences

If:

• p ≥ 0.05 → use paired t-test
• p < 0.05 → use Wilcoxon signed-rank

Step 2 (Option A): Paired t-test

Step 2 (Option B): Wilcoxon (non-parametric)

Step 3: Effect size

Scenario 3 — Comparing 3+ Groups (ANOVA or Kruskal–Wallis)

Independent groups → ANOVA or Kruskal-Wallis

Question:
Do three marketing channels differ in average revenue per user?

Data:

Step 1: Check normality of each group

Step 2: Check equal variances

Choose:

• ANOVA if normal & equal variances
• Kruskal–Wallis if not

ANOVA

Kruskal–Wallis

Effect size (eta-squared)

If significant → run post-hoc tests

Example: Tukey’s HSD (via Statsmodels)

Scenario 4 — Survey / Likert-Scale Analysis

Ordinal data → non-parametric tests

Likert data (1–5 ratings) is ordinal, not interval.

Group example:

Recommended tests

• 2 groups → Mann–Whitney U
• 3+ groups → Kruskal–Wallis
• Paired → Wilcoxon

Example

Effect size

Scenario 5 — Categorical Comparisons (Chi-Square)

Two categorical variables → Chi-square or Fisher's

Example:
Does product preference differ by age group?

Data:

Chi-square test

If expected counts < 5 → Fisher’s Exact

Effect size (Cramér’s V)

Scenario 6 — Correlation Testing

Question:
Is time spent on site related to revenue?

Step 1: Check linearity

Plot or eyeball.

Step 2: Choose test

• Pearson → linear
• Spearman → monotonic

Test

Plot