Proportion tests compare percentages, rates, or binary outcomes.

Examples:

• Conversion rate in A vs B
• Recovery rate in treatment vs control
• Pass vs fail rates
• Yes vs no survey responses

They answer questions like:

“Is the proportion of successes in one group different from another?”

Statsmodels provides:

• One-sample proportion test
• Two-sample proportion test
• Confidence intervals for proportions
• Test for equality of proportions across multiple groups

One-Sample Proportion Test

“Is the sample proportion different from a known value?”

Example:
A website has a target conversion rate of 10%. In a sample of 200 visitors, 28 converted (14%).

Example

# number of successescount = 28 # total observationsnobs = 200     # expected proportion (10%)value = 0.10    ​stat, p = proportions_ztest(count, nobs, value)print(stat, p)

Interpretation

• p < 0.05 → the observed conversion rate is significantly different from 10%
• Sign of the z-statistic shows direction (positive or negative)

Two-Sample Proportion Test (A/B Test)

“Is the proportion in group A different from group B?”

Example:

• Version A: 40 conversions out of 500 users (8%)
• Version B: 65 conversions out of 520 users (12.5%)

Example

# successes A, Bcount = np.array([40, 65])    # totals A, B  nobs  = np.array([500, 520])    ​stat, p = proportions_ztest(count, nobs)print(stat, p)

Interpretation

• p < 0.05 → conversion rates differ
• Look at group means to see which is higher

Confidence Intervals for Proportions

Statsmodels makes it easy to compute confidence intervals.

Example

from statsmodels.stats.proportion import proportion_confint​lower, upper = proportion_confint(40, 500, alpha=0.05, method='normal')print(lower, upper)

Other methods:

• "normal" — standard approximation
• "agresti_coull" — more accurate for small n
• "wilson" — recommended for general use
• "beta" — Bayesian interval

For A/B tests, you can compute CIs for each group and compare.

Multiple Proportion Test (3+ Groups)

Example:
Three marketing channels have conversion counts:

• Channel A: 30/300
• Channel B: 50/310
• Channel C: 40/290

Test whether all proportions are equal.

Example

from statsmodels.stats.proportion import proportions_chisquare​counts = np.array([30, 50, 40])nobs   = np.array([300, 310, 290])​chi2, p = proportions_chisquare(counts, nobs)print(chi2, p)

Equivalent to a chi-square test of independence, but tailored for proportions.

Practical A/B Test Example

Imagine an A/B test where:

• A: 200 conversions / 2500 visits
• B: 260 conversions / 2480 visits

Run the test

count = np.array([200, 260])nobs  = np.array([2500, 2480])​stat, p = proportions_ztest(count, nobs)print(stat, p)

Interpretation

• p < 0.05 → B performs differently than A
• Check proportions:

prop_A = 200 / 2500prop_B = 260 / 2480print(prop_A, prop_B)

Testing Directionality

By default, proportions_ztest is two-sided.

You can specify one-sided tests:

• A > B
• A < B

Example (A less than B):

stat, p = proportions_ztest(count, nobs, alternative='smaller')

Example (A greater than B):

stat, p = proportions_ztest(count, nobs, alternative='larger')

Effect Size for Proportion Differences

Cohen’s h

import numpy as np​def cohens_h(p1, p2):    return 2*np.arcsin(np.sqrt(p1)) - 2*np.arcsin(np.sqrt(p2))​p1 = 40/500p2 = 65/520​print(cohens_h(p1, p2))

Interpretation:

• 0.20 → small
• 0.50 → medium
• 0.80 → large

Cohen’s h complements the p-value by quantifying the magnitude of the difference.

Confidence Interval for the Difference in Proportions

Statsmodels does not directly compute this, but you can do it manually:

import math​p1 = count[0] / nobs[0]p2 = count[1] / nobs[1]​se = math.sqrt(p1*(1-p1)/nobs[0] + p2*(1-p2)/nobs[1])​difference = p1 - p2lower = difference - 1.96*seupper = difference + 1.96*se​print(lower, upper)​

Practical Guidelines for Proportion Tests

Use proportions tests when:

• Data is binary (success/failure)
• You compare conversion rates
• You analyse yes/no survey responses
• You compare event rates

Use chi-square when:

• Data is categorical with > 2 levels
• You work with contingency tables

Use z-tests when:

• Sample sizes are large (n > 20 per group)
• Expected counts are sufficient

Use Fisher’s Exact Test when:

• Small sample sizes
• Very low counts
• Expected frequencies < 5

Choosing the Right Proportion Test