A/B Testing Platform

Overview

A full-stack experimentation platform for running statistically rigorous A/B and multivariate tests. Goes beyond basic t-tests — implements sequential testing, multiple comparison correction, and power analysis, backed by a REST API and a React UI for managing experiments end-to-end.

What was built

• Statistical engine in Python: power analysis, Welch's t-test (unequal variance), SPRT (Sequential Probability Ratio Test) for early stopping, and Holm-Bonferroni correction for multi-variant experiments.
• Modular backend structure: backend/engine/hypothesis_tests.py, power_analysis.py, sequential.py — each concern isolated and testable independently.
• FastAPI + Uvicorn API with full endpoint documentation; SQLAlchemy 2.0 with PostgreSQL 15 for experiment persistence.
• React 18 + React Router frontend with Axios — experiment creation, result dashboards, and sequential test monitoring.
• Docker Compose deployment with separate service containers for frontend, backend, and database.
• Comprehensive README with mathematical derivations explaining the statistical methods and design rationale.

Why it matters

Most A/B testing implementations use fixed-horizon tests and ignore multiple comparison problems — both of which inflate false positive rates in practice. SPRT allows experiments to stop early when results are conclusive, and Holm-Bonferroni ensures family-wise error control when testing multiple variants simultaneously. This is the kind of statistical rigor that separates academic examples from production-quality experimentation infrastructure.