When a Cheap Demo Becomes a Commitment

You could spend the next round of work making all twelve better and leave that question unanswered. The product would be busier. You’d still have no stronger reason to choose.

But cutting ten on principle would be another guess. Twelve cheap experiments might be exactly how you discover which ideas deserve sustained work. The useful distinction is between a prototype that helps you make a decision and a prototype that acquires a permanent place on the roadmap because it already exists.

An idea can be cheap and valuable

In 2012, a proposed change to Bing’s ad headlines sat untouched for more than six months. Program managers had assigned it a low priority. An engineer eventually put it into a controlled online test, comparing the change with the existing version.

Ron Kohavi and Stefan Thomke’s account, published in 2017, reports a 12% revenue increase without harm to key user-experience metrics. The result triggered an alert because it looked too good to be true. In this case, the analysis confirmed it.

That case complicates the advice to pick a few big priorities and refuse everything else. The people ranking the idea had underestimated it. Our reading is that cheap tests deserve room to challenge a priority list, especially when the value of a change is hard to predict.

Bing had an established product, live traffic, and a way to compare outcomes. Its result gives you a reason to test an inexpensive idea, not a revenue forecast for your own. A founder still searching for buyers has a different uncertainty to resolve.

What did the demo let you decide?

Take a hypothetical reporting feature. You suspect customers would pay to replace a spreadsheet they assemble by hand. You could build several polished dashboards, but that would mainly demonstrate your ability to build dashboards. The commercial question remains: will someone pay to stop doing the work themselves?

If you can deliver the report manually from data you already have permission to use, a paid trial could test whether someone will pay for it. The sale would support that offer, at that price, with whatever help you supplied. It would leave demand for a self-service product open, along with whether you can deliver it profitably at scale.

If the buyer needs you to interpret each result, a prototype could test whether they can use the report unaided or need different information. If the value depends on immediate access to changing data, though, build enough of that path to test it. Manual delivery would miss the reason to buy.

This is where cheap tooling can work in your favor. Build several versions when comparing them will answer the same important question. Let customers try them. Preserve the option to discard the code once you’ve learned what you needed.

The trouble starts when producing a prototype automatically authorizes improving it. If each demo brings another round of customer recruitment, testing, integration, and support, cutting the coding cost has reduced only part of the obligation. If the same founder must do the customer work for all twelve, that capacity hasn’t expanded just because the demos arrived faster.

Before starting an experiment, write down the decision its result could change and the evidence you’d need to change it. Put a limit on what you’re prepared to spend finding out, including recruiting customers and evaluating what happens. A test of willingness to pay needs an actual opportunity to buy; compliments on a screen answer a weaker question.

A test with too little exposure may leave the answer unknown. Funding another round needs a reason to expect it to answer what the first one left open. If you never reached the intended buyers, recruiting them would change the evidence. Polishing the same screen would leave that gap intact.

Under those conditions, broad experimentation can serve a narrow priority. You can try many ways to help a customer finish one valuable task without accepting many unrelated product obligations.

Choose the result before ranking the work

For a business pursuing growth with enough cash to fund it, start with earning and keeping customers. Identify the purchase that a missing capability prevents, or the failure that gives an existing customer reason to leave. Then ask what evidence connects the proposed work to that outcome. Adding a revenue label to a ticket doesn’t supply the connection.

Keep the ranking conditional. If operating costs threaten your ability to finish the growth plan, reducing them can be a prerequisite. If customers leave because the product fails at its core task, an acquisition feature can wait. The category alone tells you too little to decide.

Once there’s a credible case for the benefit, compare it with the work still required to reach that result. Include customer adoption and ongoing operation, rather than stopping the estimate at a demo. A small change with measured value can deserve priority over a large initiative with a speculative payoff, as Bing’s test illustrates. A score can expose those assumptions for discussion; it can’t make uncertain inputs precise.

Choosing also means naming what will lose attention. If the same people remain assigned to every initiative, changing their order in a document hasn’t freed anyone to finish the selected work. Keep essential operations funded and assign the remaining capacity explicitly.

If customers already rely on a prototype, pausing new development still leaves a service to support. Account for that work, or agree how they’ll move off it, before treating the people maintaining it as available elsewhere.

Some stopped ideas will remain plausible. You can stop one because competing work has a stronger claim on the same people, without pretending an experiment disproved it.

For the work you stop, preserve what you learned and the evidence that would justify reopening it. The code can stay in Git without keeping a place on the roadmap.