Lesson 05 · 8 min · first-time founders

Measure what you ship

A ship date without a measurement plan is a guess. Decide, in advance, what would make you roll back.

Workshop: How to Define Products Workbook section 5 of 6 For: first-time founders

The bonus lesson CIRCLES leaves out

CIRCLES ends at “Summarize the recommendation.” In the room, that’s where the framework stops. In the real world, that’s where most products go wrong. You need to know what success looks like before you ship — and what would make you pull the change.

Pick one KPI, not five

The biggest measurement mistake first-time founders make is the same as the biggest prioritization mistake: trying to optimize for everything. You can’t. Pick one number that, if it moves, tells you the product is working. One.

If three numbers can prove your product is working, it’s probably not working.

The KPI should be specific to the customer and the user story you picked in Lessons 02 through 04. “Daily active users” is rarely the right one for an early-stage product. “Percentage of new PMs who finish onboarding in their first week” might be.

Set a baseline. Then a target.

Two numbers, written down before launch:

Baseline

What that number is today, even if you have to estimate it. Gut estimates count — better than no number.

Target

The value that would make you say “this worked.” Not aspirational — the number that, if hit, would justify shipping the next version.

The rollback principle

Decide, in writing, before launch: what would make you roll back? Not as a vague threshold — as a specific number or a specific behavior.

Why the rollback line lives in writing

Future-you, looking at a launch dashboard at 11pm with users complaining, is not the version of you who should be making this call. Present-you, calm, with the framework open, is. Write the line down now — future-you reads it.

The shape of the trigger

A good rollback trigger has three parts:

Watch the counter-metric

A KPI moving in the right direction can hide a counter-metric moving in the wrong one. The classic version: engagement goes up, but average session length goes down because users are bouncing in confusion, not staying in delight.

Pick one counter-metric. The thing that, if it gets worse, means you broke something else to make the headline number look good. Watch it as closely as the headline.

Operator story · Lesson 5

From Alon’s notebook

A feature that shipped, missed its KPI, and got rolled back. This is the lesson with the highest narrative payoff — invest the most story budget here. Suggested hook: a real moment, named metric, the room where the call got made, and what you learned about writing rollback lines in advance.

[OPERATOR STORY — Lesson 5]

Tonight’s assignment

Section 05 of the workbook. For the solution you picked in Lesson 04, write down: one KPI, today’s baseline, the target after launch, the rollback trigger (in three parts), and one counter-metric. All five lines, before any code gets written.

Assignment · Section 05

Write your measurement plan

Sign in to save your answers.

One KPI, baseline, target, rollback trigger, counter-metric. Five lines, before launch.

A number that moves when the product works. Pick one, not five.
The current value of that number, even if it’s a rough estimate.
The value that would make you say ‘this worked.’
Metric · threshold · action. All three. Decide now, while you’re calm.
The thing that, if it gets worse, means you broke something else to make the headline number look good.

Done with this section? Review all your answers in one place.

Review my workbook →
Open Lesson 06 →