WritingWhat I learned shipping an AI feature that actually gets used
PublishedJune 10, 2025
Reading time3 min read

What I learned shipping an AI feature that actually gets used

Building Contract Lens taught me more about AI product design in two weeks than a year of reading about it. Here's what surprised me.

Everyone's building AI features. Most of them are toys. The ones that stick aren't the ones with the cleverest prompts — they're the ones that fit cleanly into something someone was already trying to do.

That's the thing I kept bumping into while building Contract Lens.

The original idea was too broad

My first version tried to do everything. Upload a contract, and it would summarise it, flag risks, explain clauses, suggest edits, and generate a comparison against a template. All in one shot.

The output was a wall of text. Thorough, technically correct, and completely useless for the person who just needed to know: should I sign this?

I stripped it back to one question. Three output sections: key obligations, things to watch out for, and a plain-English verdict. Response time dropped by two thirds. User feedback immediately improved.

The lesson isn't "do less" — it's do one thing so well that the user trusts you to do the next thing.

The prompt is the product

In a traditional app, the logic lives in code. In an AI app, a lot of the logic lives in the prompt. That's a strange shift if you're used to deterministic systems.

The prompt I ended up with went through about thirty iterations. Each one taught me something:

  • Be explicit about format — if you want JSON back, say exactly what fields you expect and give an example
  • Give the model a role — "You are a legal analyst reviewing contracts for a small business owner" produces different (better) output than nothing
  • Tell it what not to do — "Do not speculate about intent, only report what is written" saved me from a lot of confident hallucinations

The final prompt is about 600 words. That's longer than most people's README files.

Latency is a UX problem

The Claude API is fast. But "fast API" and "fast app" aren't the same thing.

When I first connected the frontend to the backend, the experience felt broken even though the response time was fine. The problem was feedback — the user clicked a button, and then nothing happened for three seconds, then everything appeared at once.

Two fixes made all the difference:

  1. Streaming — use the streaming API so text appears as it's generated, not all at once. Suddenly 3 seconds felt like progress, not waiting.
  2. Skeleton UI — show the output structure (three empty sections with labels) immediately, before any content arrives. The user knows what's coming and where to look.

Both of these are standard web techniques. The AI part didn't introduce new problems, it just made existing UX problems more visible.

What I'd tell someone starting an AI project

Build the UI first. Seriously — before you touch the API, design what a perfect response looks like on screen. What does success look like for the user? Work backwards from that.

The prompt should serve the UI, not the other way around.

And ship it before it's perfect. My first "real user" found an edge case in twenty minutes that I hadn't thought of in two weeks of testing. There's no substitute for that.

← Back to writingJune 10, 2025