Ship or Kill? Statistical Deep Dive for Any A/B Test
Asked Jun 20, 2026Modified 3 days agoViewed 8 timesWorks with: GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro
0
What this prompt does
Paste your A/B test results and get a rigorous evaluation covering statistical significance, power analysis, effect size, and segment breakdowns.
- Compatible models:
- GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro
- Category:
- Data
Paste your A/B test results and get a rigorous evaluation covering statistical significance, power analysis, effect size, and segment breakdowns. Flags novelty-effect risk and common test-design pitfalls before delivering a clear ship or kill recommendation.
Analyze this A/B test: Test description: {{what was changed}} Control group (A): {{sample size and key metric result}} Treatment group (B): {{sample size and key metric result}} Primary metric: {{what you're measuring}} Test duration: {{how long it ran}} Analyze: 1. Is the sample size sufficient for the observed effect? (Post-hoc power analysis) 2. Is the result statistically significant? (with p-value and confidence interval) 3. What's the practical effect size? (absolute and relative difference) 4. Are there segment-level differences? (If segment data provided: {{list segments}}) 5. Novelty effect risk assessment (is the uplift likely to diminish over time?) 6. Recommendation: ship it, kill it, or keep testing — with reasoning. Also flag if there are any red flags in the test design (sample ratio mismatch, peeking problem, multiple comparison issues).
Source: https://sureprompts.com/blog/ai-prompts-for-data-analysis
Prompt Curator
120rep · asked Jun 20, 2026
0 Responses
Want to respond? Log in.