Visa System Design: Scalable A/B Experiment Service

Overview

You are asked to design a scalable experiment (A/B) service that deterministically assigns users to variants for features at runtime. The frontend calls an endpoint with a userId and featureName and expects a stable variant (e.g., "control", "treatment_A"). Your design must support multiple independent experiments, admin overrides, logs for metric collection, and easy experiment lifecycle management.

Flow & stages

1. Request: frontend calls GET /assign?userId=123&featureName=new_ui
2. Resolve experiment configuration (is feature enabled? active experiment, variant weights)
3. Check overrides or manual enrollments for the user
4. Compute deterministic bucketing (consistent hashing on userId+featureName)
5. Return variant and asynchronously log the assignment for metrics

Key components to propose

• Config store for experiments (multi-tenant schema, versioned configs)
• Fast read path: cache (Redis) with eventual consistency and fallback to config DB
• Deterministic bucketing: consistent hash + normalized cumulative weights
• Overrides service: small, strongly consistent store for per-user overrides
• Logging / event pipeline: write assignment events to a durable stream (Kafka) for downstream analytics

Non-functional & trade-offs

Design for <100ms P95 reads with horizontal scaling: use in-memory caches near the API, shard configs for large scale, and replicate read-only config snapshots for HA. Consider eventual consistency for config changes vs. immediate rollback needs. Plan for graceful degradation (return default variant if config store is unreachable) and strong audit logs for experiment integrity.

Skill signals

You should demonstrate understanding of distributed systems (caching, sharding), data modeling for multi-tenancy, deterministic hashing algorithms, API design, and how to integrate logging/metrics for experiment analysis.