03_sales_enablement/testing_log/ab_test_log
A/B test log
Sales-ability is built by testing, not by guessing. Every meaningful variant goes here.
Open hypotheses (from prior report)
| Test area | Variant A | Variant B | Hypothesis | Success metric |
|---|---|---|---|---|
| Top-line message | "Standardise the firmware foundation" | "Stop funding repeated firmware rebuilds" | Finance-led language outperforms architecture-led language with CFO/CEO | Positive reply rate |
| Pilot label | "Proof of concept" | "Migration economics pilot" | Outcome language converts better than experiment language | Meeting-to-proposal rate |
| Primary wedge | Matter / Thread migration | KNX IoT / CRA readiness | European building-automation buyers respond more strongly to KNX/CRA | Qualified-opportunity rate |
| Economic framing | Hard savings only | Savings + launch-delay risk | CEOs respond more to timing risk than CFOs do | Persona-specific conversion |
| Sender | Founder | Commercial lead | Founder credibility outperforms in early-market accounts | Reply quality |
| Asset after first meeting | ROI worksheet | Procurement pack | Finance buys the math; procurement buys the friction reduction | Proposal acceptance speed |
| Pilot pricing cue | Fully credited assessment | Partially credited assessment | Full credit may help early trust; partial credit may preserve seriousness | Assessment-to-pilot conversion |
| Target account size | 50–250 staff | 250–1,500 staff | Mid-market buyers move faster; larger buyers produce bigger pilots | Sales-cycle length, ACV |
Active tests
| Test ID | Started | Variant A | Variant B | Sample size | Status | Result |
|---|---|---|---|---|---|---|
Concluded tests
| Test ID | Concluded | Winner | Confidence | Action taken |
|---|---|---|---|---|
Test discipline
- Pre-register every test (variant, hypothesis, metric) before running
- Minimum sample size: 30 per variant before reading results (more for low base-rate metrics)
- Don't peek and pivot — run to completion
- Always write the learning, not just the winner
Last modified 2026-05-05. Suggest a change to this page →