Track and iterate

The experimentation loop and the review of results.

Sailboat under full sail at sea

The audit keeps its value if three things exist after the presentation: one scoreboard, an experiment log and a monthly review that can change the plan. Without them, the roadmap drifts within a couple of months and the team goes back to shipping whatever is loudest.

I’ll stay with the fictional B2B SaaS at 2M ARR from the six-month roadmap. Every figure about it below is illustrative.

Keep one scoreboard tied to the growth equation

Take the metrics from your growth equation, and stop at five to seven. Each one gets a single source and a single owner, so nobody argues about whose number is right.

Metric Source Owner Baseline Month 3 target Month 6 target
Qualified leads per month CRM Head of growth 25 35 50
Signups with a known first-touch source Acquisition analytics Head of growth 40% 80% 90%
Visits to feature and pricing pages from the blog Acquisition analytics Content lead 800 1,500 2,500
Signup to activated Product analytics Head of growth 30% 33% 38%

Qualified leads sit at the top on purpose. Signups can rise while the people signing up are the wrong size for you, so use signups as an early signal and decide on qualified leads.

Read by cohort of signup month, because a total mixes old and new behavior. This query assumes a signups table with the dates of each stage:

SELECT date_trunc('month', s.created_at) AS signup_month,
       s.first_touch_source,
       count(*) AS signups,
       count(*) FILTER (WHERE s.activated_at IS NOT NULL) AS activated,
       count(*) FILTER (WHERE s.qualified_at IS NOT NULL) AS qualified
FROM signups s
WHERE s.created_at >= date_trunc('month', now()) - interval '6 months'
GROUP BY 1, 2
ORDER BY 1, 5 DESC;

The latest cohorts will look weak because their leads have had less time to qualify. Compare cohorts at the same age. And write down the date of every tracking change: when UTMs arrive on blog CTAs, “direct” will drop and “blog” will climb without anything having happened in the market.

Turn each action into a written hypothesis

Every roadmap line that can succeed or fail gets a card, written before launch. The decision rule matters most, because writing it after the result lets anyone read the result as they like.

Field Example
ID EXP-014
Hypothesis Adding a CTA to the matching feature page after the first section of our 20 most-read articles raises qualified leads from the blog, because today’s CTAs sit at the bottom
Main metric Qualified leads from the blog, by first touch
Guardrail Time on page and signups from other sources stay stable
Method Before and after by cohort, on those 20 articles
Duration Eight weeks, set in advance
Decision rule At least +30% over the prior eight weeks: roll out to all articles. Below +10%: revert and rework the offer
Result and decision Filled in at the review

Tag each test so that it can be isolated in your reports:

?utm_source=blog&utm_medium=cta&utm_campaign=exp-014_feature-cta&utm_content=after-section-1

Log the failures as carefully as the wins. After a year, the file of what did not work is worth a lot to the next person who proposes the same idea.

Small boats on a calm grey horizon

Check what your volume can actually prove

A/B testing needs traffic that many B2B companies don’t have. Before launching one, estimate the sample with a common approximation, for a two-sided test at 5% significance and 80% power, where p is the baseline rate and d the absolute difference you want to detect:

n per variant = 16 * p * (1 - p) / d^2

Say the SaaS gets 400 signups a month and activates 30% of them.

Smallest lift to detect Per variant Total Months at 400 signups
30% to 35% 1,344 2,688 6.7
30% to 40% 336 672 1.7

A five-point lift would eat the entire program for a single test, so this team only A/B tests changes big enough to move ten points. Everything else goes through weaker methods, stated as such: before and after by cohort, rollout to one country or segment first, or customer calls and session recordings. Set the duration in advance and don’t stop when the curve looks good, since checking repeatedly and stopping on a favorable moment inflates false positives.

Lone sailboat on open water

Hold a review that can change the plan

Rhythm Length Who Output
Weekly 30 minutes Action owners What shipped, what is blocked
Monthly 60 minutes Owners and founder Scoreboard read, decisions on finished experiments
End of month 3 Half a day Owners and founder Scores updated, plan for months 4 to 6 reworked
Month 6 One to two days Whoever ran the audit Light re-audit, next plan

The monthly review always ends with one of these decisions per finished experiment.

Result Decision
Rule met, guardrails fine Roll out and make it the new standard
Rule met, guardrail broken Don’t roll out, look at the segment that lost
Flat Stop. One retry at most, with a different hypothesis
Worse Revert, and record why you think it failed
Inconclusive, too little volume Decide on judgment and label it as judgment, or test something bigger

Read rising curves as critically as falling ones. On Derrick, the last 12 months brought 42.4K clicks and 19.2M impressions, which looks great until you see the 0.2% click-through rate: the site gets seen far more often than it gets chosen, and that gap is the next piece of work.

At month 6, run a light re-audit

Fix a bottleneck and it moves, so the verdict from six months ago is probably out of date. Replay the short version of the audit, and start with the data, since a new tool or a site change can have broken something along the way.

Check Question to settle
Tracking Does each conversion still have a source? Has the multi-touch model replaced last click in decisions?
Funnel Which stage now has the worst rate, and for which segment?
Parked list Which reopening conditions are met?
Experiment log Which kinds of action worked here, and which never did?
Capacity Did the team deliver what the plan assumed? If not, cut the next one

From this comes a new one-page verdict and a new six-month plan, built the same way as the first, with real results replacing most of the estimates. The first plan will have been too ambitious somewhere, and now you know where.

Your next step: create the scoreboard this week with the baselines you have today, put the monthly review in everyone’s calendar for the next six months, and write the card for the first experiment before it ships. A baseline can’t be reconstructed once the change is live.