Share with your CTO
The dominant assumption in enterprise AI, that the newest frontier model is always the right default, is breaking down under real operating conditions. Writing from their experience at Campfire, an AI-native accounting platform, the authors describe a pattern that’s appearing across back-office workflows: probabilistic reasoning, which works well for creative tasks, becomes a liability when the system needs to produce the same correct answer 500 times in a row. Their fix, having the model generate deterministic code rather than reason directly toward an answer each time, cuts unpredictability at the cost of some flexibility. A domain-specific smaller model they post-trained now slightly outperforms leading commercial models on their core tasks at roughly 2 percent of the inference cost.
What this means for your business
The organizations most exposed here are those that built their AI roadmap on a single escalator: wait for the next model, upgrade, repeat. That approach made sense when AI lived in assistive, low-stakes workflows. It starts to break when AI touches records, payments, policy enforcement, or anything that feeds an audit. If your deployment runs millions of inferences against tightly bounded operations, the cost math alone forces a rethink before any capability argument does.
The code-generation detour the authors describe is worth unpacking because it reframes what “AI doing the work” actually means. Instead of passing financial data to a language model and trusting it to reason freshly each time, the model writes a SQL query or Python script once, that code gets validated, and then deterministic software runs it repeatedly. The model’s creativity is spent upfront on rule-translation, not on every individual transaction. This is less a workaround than a principled separation of concerns: AI handles the ambiguity of encoding business logic, conventional software handles the execution that demands consistency.
The who-wins call here isn’t between large and small model vendors. It’s between enterprises that build tiered model stacks matched to task type and those still treating frontier model access as a proxy for AI maturity. The Campfire result, 2 percent of inference cost with slightly better domain accuracy, will look more compelling the moment a CFO starts scrutinizing per-transaction AI costs at scale. CTOs who haven’t yet mapped their AI workloads by required consistency, latency, and explainability are building the justification problem now, not later. I’d revise this view if domain-specific fine-tuning proves too brittle to maintain as business rules shift, but the inference cost pressure alone makes the tiered approach worth stress-testing.
Concept deep-dive: Deterministic versus probabilistic execution
A probabilistic system, like a large language model, doesn’t follow a fixed script. Given the same input twice, it can take a slightly different reasoning path and land on a different output, the way two experienced analysts might word the same conclusion differently. A deterministic system, like conventional code, produces exactly the same output for the same input, every time. In enterprise workflows that feed compliance records or financial statements, “close enough” isn’t a passing grade, which is why the architecture choice between these two modes has direct audit and liability consequences.
Based on reporting from Why Smarter A.I. Models Aren’t Always Better for Business, originally published 2026-10-09 09:45:00.
