A/B Testing & Experimentation
Design, run, and interpret experiments that drive product and marketing decisions.
About this course
A rigorous A/B testing course covering the full experimentation lifecycle: hypothesis formulation, sample size calculation, implementation, analysis, and decision-making. You will learn to avoid the most common mistakes (peeking, multiple testing, novelty effects) and build an experimentation programme that produces reliable results.
Target audience: Product managers, growth marketers, data analysts, developers running experiments
What you will learn
- A/B testing
- Statistical significance
- Sample size calculation
- Experimentation programme design
- Causal inference basics
Course syllabus
10 modules · video + exercises
- 1Why A/B testing: the value of causal inference in product decisions
- 2Hypothesis formulation: from idea to testable prediction
- 3Metrics: choosing the right primary and guardrail metrics
- 4Sample size and power: how long should a test run?
- 5Randomisation: unit of randomisation and assignment methods
- 6Running the test: implementation, instrumentation, and QA
- 7Statistical analysis: t-tests, chi-square, and confidence intervals
- 8Common mistakes: peeking, multiple comparisons, and SUTVA violations
- 9Interpreting results: when to ship, iterate, or abandon
- 10Building an experimentation programme at scale
Prerequisites
- –Basic statistics
- –Familiarity with web analytics
Frequently asked questions
Which tools does this course use?
Exercises use PostHog (open-source) for experiment setup and Python for statistical analysis. The principles apply to Optimizely, VWO, LaunchDarkly, or any other platform.
Ready to start A/B Testing & Experimentation?
Join 3,700+ learners already enrolled. Self-paced, certificate on completion.