The third post in this series is about the two steps from the pre-build process most operators skip when they’re under deadline pressure. They’re the cheapest insurance the process buys. They’re also the easiest to cut, because by the time they’re due, the project feels like it’s ready to start.
It isn’t. Or, more precisely, you don’t know yet.
What each one catches
The what-if audit and the pre-mortem look superficially similar — both are exercises in imagining what could go wrong before any code runs — but they catch different things.
The what-if audit points an external auditor at the locked whitepaper and asks: what’s wrong with this brief, structurally? The auditor’s job is to find ambiguities, defensible-looking assumptions that aren’t, missing pieces, and scope risks. The output is a punch list of edits to the whitepaper. The whitepaper gets tighter.
The pre-mortem points the operator at the project as a whole and asks: imagine this has shipped, run for six weeks, and is being shut down as a failure — walk through every plausible cause. The output is a list of failure modes to defend against, baked into the architecture or the schedule before any code begins.
The audit is a brief-quality test. The pre-mortem is a project-quality test. The audit catches “the document doesn’t say what it needs to say”; the pre-mortem catches “the document says everything it should but the world will still end the project for reasons you didn’t write down.”
Run only the audit, and the brief is sound but the project still has unaddressed structural risks. Run only the pre-mortem, and you’ve identified the risks but the brief still has the ambiguities that will cost time during the build. Run both, in this order, and the whitepaper becomes a contract that’s also been stress-tested against the world.
The what-if audit, in practice
The discipline that matters: a different model than the one that helped you draft the whitepaper. Same model, same blind spots. The point of an external audit is to surface the things the drafting context could not see.
I open a fresh chat, with a different LLM than I used during drafting. I prime it with a strict role: senior systems engineer, no investment in the project, paid for honest review. I paste the whitepaper. I ask for output in a fixed structure — ambiguities, defensible-looking assumptions that aren’t, missing pieces, scope risks, the three most plausible failure narratives, and a punch list of whitepaper edits ordered by importance.
The output is rarely “this looks great.” When it is, that’s a signal the role wasn’t strict enough or the model isn’t critical enough — try a different model, or rewrite the prompt.
The output is usually 6 to 12 specific edits. Most go into the whitepaper. Some kill the project before any code runs. Both are acceptable. A project killed at the audit phase is a project that didn’t burn a quarter of effort proving what one hour of stress-testing would have surfaced.
The pre-mortem, in practice
I sit down with the locked-and-audited whitepaper and run a single mental exercise: it’s six weeks from today. The project shipped four weeks ago. It’s now being shut down as a failure. I’m writing the postmortem.
For each plausible cause, I ask:
- What in the whitepaper should have prevented this?
- What in the architecture should have prevented this?
- What in the schedule, the alerting, the operations plan should have prevented this?
Causes I work through, every time, in roughly this order: external dependency failure (vendor down, API quota exhausted, contract changed under me), scale failure (works at 10 records, breaks at 10,000), data quality (input is wrong but the system trusts it), people (operator unavailable, no docs), timing (incident during launch week, no rollback path), hidden assumptions (about user behaviour, market, technology).
For each plausible cause that has a real prevention I haven’t yet built, the prevention goes into the whitepaper now. Not after the actual postmortem.
The pre-mortem usually surfaces 3 to 6 architectural changes. Some are small (an extra fail-fast on quota errors). Some are large (a structural change to how a per-record loop interacts with a rate-limited API). Either way, baked in before any code, they cost a fraction of what they’d cost as retrofits.
The pattern I’ve seen most often
The expensive bug I caught in the trading-bot project — a strategy that printed +18.4% in backtest and −3.1% in honest paper-trade — was caught by a single architectural decision: the paper-trade gating layer was built before any live-trading code, and live-trading code was structurally gated behind it. That gating layer was the project-level result of a pre-mortem. Without it, the strategy would have shipped on the strength of a backtest that was lying to me.
The pre-mortem produces patterns like that. A small architectural decision, baked in before any code, that sits there quietly until the day it’s the only thing standing between you and a real loss.
Why both stress-tests are the first to be cut
Both run in roughly an hour each. Both produce concrete output. Both have, on every project I’ve run them on, surfaced something I would have missed.
They’re still the first thing cut when projects are under deadline pressure, for one specific reason: the cost of skipping them is invisible in the moment.
When the build starts on time without an audit and without a pre-mortem, nothing immediately goes wrong. The build proceeds. The team feels productive. The shipped product has bugs that, with a half-hour audit and a half-hour pre-mortem, would have been preventable. But those bugs surface during operation, not during planning, and the project’s accounting never connects them back to the skipped steps.
I’ve stopped letting myself negotiate this trade. The audit and the pre-mortem run on every project that takes more than a long weekend. They’re not optional; they’re not “if there’s time”; they’re the gating step between the whitepaper being written and the build being allowed to start.
The asymmetry is the point. The cost of running them is a deterministic two hours. The cost of skipping them, for a project of any reasonable scope, is unbounded.