
What shipping AI features actually taught me
Draft starter post — replace with your own story. It’s here so the blog isn’t empty on day one.
Demos are easy. Production is where AI gets humbling. Here are a few things I keep relearning.
1. The model is the easy part
The hard parts are everything around it: clean inputs, sensible fallbacks when the model is wrong, latency budgets, and a UX that sets the right expectations. A great prompt with a bad interface still feels broken.
2. Evals beat vibes
“It feels better” doesn’t scale. A small, honest evaluation set — even 30 hand-labeled examples — tells you more about a change than an afternoon of clicking around.
3. Design for being wrong
Users forgive an AI that’s occasionally wrong if it’s easy to correct and never destructive. Make the undo obvious, keep a human in the loop where stakes are high.
4. Speed is a feature
A correct answer that takes ten seconds often loses to a good-enough answer that’s instant. Stream early, cache aggressively, and trim the round-trips.
These are the notes I wish I’d had earlier. I’ll go deeper on each in future posts.