Your activation metric might be a thermometer, not a lever

Every growth team eventually goes hunting for its “aha moment” — the early action that separates the users who stick from the users who evaporate. You pull the data and there it is, gorgeous: people who added three collaborators, or sent twenty messages, or connected an integration in week one retain at triple the rate of everyone else. It goes on a slide. Onboarding gets rebuilt to shove every new user toward that action. And six months later, retention hasn’t moved.

Here’s what went wrong. You found a number that correlates with retention and treated it as a number that causes retention. They are rarely the same number. The behavior you spotted might genuinely be the aha moment — the thing that makes the product click into place. Or it might just be what already-committed users happen to do on their way to being committed. A thermometer reads high when the room is hot; it does not make the room hot. Grab the thermometer and force the reading up — drag every user by the collar into adding three collaborators — and you’ll often move the metric while the thing underneath it (did they have any actual reason to invite three people?) stays exactly where it was.

The confound has a shape, and once you see it you can’t unsee it.

This is not the same mistake as gaming a metric after you’ve chosen it — that’s Goodhart’s law, the number decoupling from the goal once it becomes a target. This is the earlier, quieter error: choosing the wrong metric to chase in the first place, because a clean correlation on a dashboard is so much cheaper to obtain than an actual cause.

How to tell a lever from a thermometer. Prefer variation you didn’t engineer. Did users who tripped the aha event for incidental reasons — a UI change, a new default, a fluke of navigation — retain like the ones who sought it out? If forced and organic users retain the same, you may have a real lever. If only the organic ones retain, you’re holding a thermometer. Run the holdout. The honest test of “onboarding should push the aha event” is an experiment: some users get the push, some don’t. If the pushed cohort retains better, it’s causal. If they hit the event far more often but retain the same, you’ve been buffing a thermometer. Check the timing and the obvious confounder. A genuine cause should precede the lift and survive controlling for plain engagement — because if your “aha event” is really just used the product a lot in week one, you’ve merely rediscovered that engaged users are engaged. Then pick the earliest event that survives all of that. Among the behaviors that are actually causal, the one that happens soonest is the most useful lever — you can still influence it before the user is gone.

None of this means activation metrics are useless. A validated one is the single best focus a growth team can have. It means the finding is the beginning of the work, not the end of it. Correlation tells you where to look. Only an experiment tells you where to push.

The dashboard will always hand you a beautiful correlation for free. Charging it rent — proving it’s a lever before you rebuild the whole funnel around it — is the unglamorous work that separates growth from motion.

The concepts, to look up: activation / “aha moment” metrics; correlation vs causation and confounding (the third-variable problem); the North Star metric debate; holdout experiments and natural experiments as causal tests. Distinct from Goodhart’s law and survivorship bias in metrics.