Creative Ventures engineering9 min read
How to estimate AI features: a practical framework for product teams
AI feature estimation is harder than CRUD sizing — the happy path lies. A three-axis framework for AI feature scoping: accuracy tolerance, recoverability and data exposure.
Estimating a CRUD feature is a habit. Estimating an AI feature is an argument. The model works in the demo, fails 8% of the time in real use, and the 8% is exactly where your users are. Here is the framework we now use before we give a client a number on any AI feature.
The three axes we measure in AI feature estimation
Every AI feature we are asked to scope gets sized along three axes. Accuracy tolerance — how wrong can the output be before the user cares. Recoverability — if the model gets it wrong, what does recovery cost. Data exposure — what does the model need to see to do its job, and what is the blast radius if it leaks.
What we used to get wrong about AI estimation
Our first year of AI estimates were basically software estimates plus a fudge factor. We scoped the happy path, multiplied by 1.5 and called it a day. We consistently missed the eval harness, the fallback UI and the human-in-the-loop path. None of those are optional in production; all of them are invisible in a demo.
“The cost of an AI feature is the cost of the happy path, times the cost of the recovery path.”
The one-page template we use for every AI feature
Every new AI feature has a one-page doc: task definition in one paragraph, accuracy floor as a single number, fallback UI in two sketches, human-in-the-loop path as a diagram, data footprint as a bullet list. If any of the five is hand-waved, the feature is not ready to estimate.
