Add A/B testing docs, enhance test suite and environments pages

[mintlify]

Summary

Documentation updates based on insights from the Builder Pod Notion page, cross-referenced against Notion specs and engineering repos.

Changes

• New page: A/B testing - Documents the canary traffic splitting feature shipping in Q1 2026. Covers how it works, starting/stopping tests, reviewing results, and best practices.
• Enhanced test suite page - Added AI-generated test cases section documenting the llm_testify feature.
• Improved environments page - Added practical guidance for different deployment scenarios. Addresses customer pain point around 4-environment confusion.
• Updated navigation - Added A/B testing page under Deployments > Environments.

TODOs for team review

• Confirm whether A/B testing traffic split is fixed at 50/50 for MVP or supports custom percentages
• Confirm exact filter name for A/B test group in Conversation Review UI
• Confirm llm_testify rollout status

Sources

• Builder Pod Notion page
• A/B Testing in Experimental Config (Notion spec)
• AS Testing Suite HLD (Notion spec)
• ab_test_id references in poly_core
• llm_testify feature flag in platform_ui

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
polyai	🟢 Ready	View Preview	Apr 10, 2026, 10:19 AM