Experiments beat assumptions

A product idea is a hypothesis until someone's behaviour says otherwise.

A working rule. Not an absolute.

Entry updated 26 Oct 2026

I can be convinced by my own explanation of a product. That explanation is not demand. Demand shows up when someone tries to use the thing, pays for it, or changes how they work. Until then I am holding an assumption.

A useful experiment is small enough to run and sharp enough to surprise me. It names what I expect people to do, and it counts that behaviour rather than praise, visits, or how finished the interface looks.

Building the product and validating the demand are different jobs. I have felt that gap while making software: the build can go well and still leave the question of distribution unanswered. Shipping is not the same as being chosen.

After a test, the decision is to continue, change direction, or stop. Stopping is a result. It is not a failure of nerve if the behaviour did not show up.

Influences

The Lean Startup, by Eric Ries, is the book behind this language of hypotheses and feedback.

How I apply it

Write the assumption in one sentence before building the next slice.
Prefer a behaviour I can observe over a metric that only shows activity.
Keep a note of what would make me stop, before I am attached to the build.

What I can actually point to

I have several public products and several ideas that are not products. I have not published conversion numbers, revenue, or a claim that a particular experiment succeeded. The record on this site is the existence of the work, not its market result.

Where this could be wrong

Some systems, especially in finance and safety, should not be 'tested' on people casually. The experiment has to match the cost of being wrong.
A tiny test can also lie. The wrong audience, or a test that only measures curiosity, can be mistaken for demand.

What could change my mind?

A situation where more building, before any test, was clearly the responsible path because the assumption could not be probed any other way.
Evidence that my experiments are too small to teach me anything I did not already believe.