After Vibe Coding: The Work a Demo Leaves Behind

Stripe’s engineering account of this failure mode explains the ambiguity: an operation can succeed even when the caller never receives its result. If you prompted a payment flow into existence without tracing its failure branches, you have an additional problem. Before choosing a repair, you have to discover what your own code does next.

If AI made it possible for you to build the product at all, that gain counts. Your alternative may have been no product, rather than the same product written by an experienced engineer. Keeping it running adds work that the successful demo could leave unfinished. Some of that work is fixing bugs. Some is learning enough to recognize a proposed fix that would make things worse.

Where the payment can be repeated

Stripe’s account describes a remedy: an idempotency key, a unique identifier that lets the provider recognize a repeated request. For an immediate retry of the same request, sending the same key lets Stripe return the saved result if the operation already completed.

Your app has to preserve that identity. If its retry generates a fresh key and requests a fresh payment, Stripe has no reason to recognize it as the original request. A new random string on every attempt can look perfectly tidy in a code review. It also defeats the reason the key exists.

For the operation that creates the payment, save that key with the local purchase before sending the request. A restart then has an identity to recover. Even a saved key has limits: Stripe’s API reference says keys can be removed once they’re at least 24 hours old, and reuse after removal creates a new request. A delayed recovery needs to establish what happened to the original payment before creating another.

Follow the payment one step further. A webhook is a notification Stripe sends to your app, for example to report that a payment succeeded. Stripe’s integration guide, published in February 2025, warns that the same event can arrive more than once. Receiving another notification doesn’t mean another purchase occurred.

Suppose your product sells usage credits. If the handler adds credits every time it receives a successful-payment notification, replaying that notification would add the credits again, even with only one payment in Stripe. Preventing a repeated payment request and preventing repeated credit allocation require checks in different places.

Now ask what happens if the app stops after updating the customer’s balance but before recording that it handled the notification. On replay, it finds no record and adds the credits again. Reversing the writes leaves the opposite failure: the app can mark the work done, stop before granting credits, then skip the replay. The customer paid either way.

If both records share a transactional database, they can be saved in a single all-or-nothing transaction. An interruption before it commits leaves neither update; a completed transaction saves both. That closes the gap between those writes.

Even then, imagine duplicate notifications being processed at the same time. Each handler could check for a completed grant before either has saved one. For this purchase, a database uniqueness constraint can refuse a second grant record carrying the same required purchase ID. Insert that record and add the credits in the same transaction. If the record already exists, the handler must leave the balance alone.

The purchase ID matters because it identifies what the customer bought. Stripe also documents duplicate events with different event IDs, so checking only the notification’s ID can miss a duplicate. For this example, every notification that would grant those credits has to resolve to the same purchase.

The transaction protects the local database. It doesn’t reverse a payment already accepted by Stripe, so the provider’s retry behavior still needs its own check. If granting value involves another external service instead of a balance in that database, the local transaction leaves that service’s behavior unproven too.

These conditions can be tested before launch. They don’t require a traffic surge, and a model can help write the tests. Someone still has to decide that a lost reply, a repeated notification, and an interrupted update belong in the test at all.

The comprehension tax

In that payment example, the extra work begins before the patch. You have to reconstruct the relationship between the customer’s purchase, Stripe’s payment, your credit balance, and the record of a handled notification. Reading the function that raised an error won’t necessarily reveal all of it.

That’s the comprehension tax: the work of reconstructing how a system behaves before you can safely change it. Generating code can defer that work when you accept the output without checking how it fits together. If you keep adding behavior on that basis, a later repair has more unexamined connections to follow. This is a consequence of the workflow, not a measured failure rate for AI-built products.

Handwritten software can leave you with the same work. A developer who reviews and tests generated code can build understanding while using AI. Simon Willison made that distinction in March 2025: he separates accepting code without reviewing it from AI-assisted development in which the developer reviews, tests, and can explain the result.

Code quality and your understanding of the code are separate questions. A correct payment integration can still be hard for you to change if you don’t know why it reuses one identifier and creates another. Replacing it with cleaner-looking generated code could discard the very behavior keeping the payments safe.

You don’t need to memorize the repository. You need an explanation you can check for the customer action you’re about to change. For payments, that explanation crosses the boundary between your app and Stripe. For a private dashboard, it crosses the boundary between login, data retrieval, and any saved copy of the page. A successful login proves very little about privacy if the app then serves every account the same cached dashboard.

Work through one customer action

Start with the customer action and its expected result. For this purchase, retries must not create another payment, and recovery must add the purchased credits once. That expectation gives you something to compare with the code, even before you can explain the implementation.

Ask the model to trace the existing implementation before proposing edits. Have it point to where the purchase is identified, where the payment result is stored, and what tells the app that the credits have already been granted.

Treat the explanation as a set of claims to check. If it says retries are safe, find the identifier the retry actually sends. If it says an interrupted update recovers, identify the stored record from which recovery begins. A fluent explanation that skips the gap between two writes gives you another question, not permission to deploy.

The tests deserve the same skepticism. If a generated test copies the implementation’s assumption that each notification earns credits, it can approve the very duplicate you’re trying to prevent. Its expected balance has to come from the purchase, rather than repeating the handler’s calculation.

Make the same path visible when it runs. Attach the local purchase ID to the payment’s metadata when creating it, so a Stripe record can be matched back even if your app loses the reply. Record the provider’s payment ID and the notification your app handled too, without logging card data or secrets.

Those records should let you distinguish a second payment from a second notification of the first. An uptime monitor can’t answer that question; a running server can still allocate the wrong balance.

If an engineer on your team can trace and exercise that path with you, use that existing knowledge. For a low-stakes prototype you can discard, you may reasonably stop short of production safeguards. Where real money or private data is involved and nobody can evaluate the behavior, a capable peer or a focused external review can supply the missing expertise. Ask for a demonstrated failure path and a checked repair. A screen-share alone can’t establish that the rest of the product is safe.

Then use a separate test environment to let Stripe complete a payment while withholding the reply from your app. Confirm that original success in Stripe’s test records, then retry through your app’s own retry path. Compare the provider’s payment with the local purchase and credit balance.

Replay the successful-payment notification, both sequentially and with handlers running at the same time. Confirm that the handler accepts the replay: a rejected test request proves nothing about duplicate handling. The balance must stay at the amount justified by the purchase.

Interrupt the local update between the balance write and the record marking the purchase fulfilled. After restart and replay, confirm that the purchased credits were added once and the completed-grant record agrees. Keep these reproductions as regression tests once the behavior is right.

If a later generated refactor removes the duplicate protection, those tests should fail even if an ordinary test payment still succeeds.