Calculate Statistical Power for Experiment Design

Task: Calculate Statistical Power for A/B Testing

Statistical power is a crucial concept in experiment design. It represents the probability of correctly rejecting a null hypothesis when there is a true effect (i.e., detecting an effect when it exists). Power analysis helps researchers determine the appropriate sample size needed for their experiments.

In this task, you will implement a function to calculate the statistical power for a two-sample z-test, commonly used in A/B testing scenarios.

Your Task:

Implement the function calculate_power(effect_size, sample_size_per_group, alpha, two_tailed) that:

1. Takes the following parameters:

   • effect_size: Cohen's d (standardized effect size), the difference between group means divided by the pooled standard deviation
   • sample_size_per_group: Number of observations in each group
   • alpha: Significance level (default: 0.05)
   • two_tailed: Boolean indicating if the test is two-tailed (default: True)

2. Returns the statistical power as a float rounded to 4 decimal places

Notes:

• You may use the math library for mathematical functions
• The standard normal CDF can be computed using the error function: CDF(x) = 0.5 * (1 + erf(x / sqrt(2)))
• For the inverse CDF (quantile function), you may use a numerical approximation

import math

def calculate_power(effect_size: float, sample_size_per_group: int, alpha: float = 0.05, two_tailed: bool = True) -> float:
    """
    Calculate statistical power for a two-sample z-test.
    
    Parameters:
    effect_size: Cohen's d (standardized effect size)
    sample_size_per_group: Number of observations per group
    alpha: Significance level (default 0.05)
    two_tailed: Whether the test is two-tailed (default True)
    
    Returns:
    Statistical power as a float rounded to 4 decimal places
    """
    # Your code here
    pass