A practical guide to shipping AI features that stick

Every AI feature we've shipped that actually stuck around shares three traits: it does one narrow thing, it was evaluated against real examples before launch, and it fails visibly instead of confidently.

Narrow beats general. 'Draft a reply to this support ticket' ships and gets used. 'AI assistant that helps with anything' gets a demo and then silence, because nobody knows what to ask it.

Evaluation isn't optional. Before any AI feature reaches a user, we run it against a reference set of real (anonymized) examples and score the output against a rubric a domain expert — not an engineer — signed off on.

And failure has to be visible. A model that's uncertain should say so, or defer to a human, rather than confidently generating something plausible-sounding and wrong. That one decision is the difference between a feature people trust and one they quietly stop using.