How we made a sales-call AI assistant production-ready, and what the same thinking did for a refunds and chargebacks agent. A companion to the Cutting LLM Costs series: that series is about what an assistant costs, this one is about what keeps it safe.
The first rule we wrote for the sales assistant had nothing to do with prompts.
The agents could read from any system they needed. They could write to none of them.
That rule took an afternoon to agree on, and it did more for reliability than any prompt change we made in the months after. This post is about why, and about the other layers we put around the model.
What we were building
The client (I can't name them) wanted sales reps and account managers to have product context before, during and after customer calls. We built a RAG assistant to do it. It was multi-agent: different agents read from databases and other services, pulled what mattered, and handed it to the rep. I wrote up the full build in Building a RAG and AI Agent Assistant That Helps Sales Teams Close Faster and the memory design in The Blueprint of AI Agent Memory.
It ran on Claude 3.5 Haiku and Claude 3.5 Sonnet, the generation we built on. Newer models have shipped since, and nothing below depends on which one you use.
The model was never the hard part. The hard part was answering one question for the client: what is the worst thing this system can do on a live call, and what stops it?
Control 1: Read-only by design
Every agent was read-only. If an agent hallucinated, the damage stayed inside the answer. It could not change a deal, edit a record or send anything on its own. A bad answer is a problem. A bad action is an incident. We chose to only ever have the first kind.
This is a deterministic control. It doesn't depend on the model behaving well. It depends on the permissions the agent was given.
Read-only is necessary, not sufficient. Two questions sit behind it. What can the agent read? Reads should be scoped to the rep, the account and the region, so the assistant never shows a rep what they shouldn't see. And what does it do with what it reads? An agent that pulls from many systems also pulls in text anyone can write, such as a CRM note or an uploaded document, and that text can carry instructions. Treat everything retrieved as data, never as a command. Read-only is what limits the damage if one slips through.
Control 2: A golden dataset that blocks releases
We built a golden dataset of 300 examples and tested against it on three things:
- Accuracy. Did the assistant give the right answer?
- Stale answers. When the business logic changed, did the answers change with it, or did the assistant keep repeating the old rule?
- Hallucination against policy. Did it say anything the company's policy contradicts?
The second one is the one I'd underline. I call it staleness, not model drift: the model didn't change, the business did. Sales logic changes: pricing, eligibility, terms. An assistant that was right last quarter can be confidently wrong this quarter. The golden dataset is how we caught that before a rep did.
Here is the kind of case those tests exist for (an illustration, not a client record). A pricing change takes effect on the first of the month: reps can now offer up to 15% off annual renewals without approval, up from 10%. The golden dataset has a question for exactly this: "What discount can I offer on an annual renewal without approval?" The expected answer is 15%, per the pricing policy updated this month.
After the change, the old pricing page is still in the index, and the assistant answers 10%. It cites a real document and sounds certain, and it is wrong. That question fails the accuracy check, the release is blocked, and the fix is a content one: retire the old page and re-index. Without that question, a rep finds out on a call, when the customer asks for 12% and the rep says no to a discount they were allowed to give.
A dataset only blocks releases if it is run like one. That means stratifying it by feature, product and risk tier so a gap shows up where it hurts. It means a named owner, a refresh every time pricing or policy changes, and a written rule for questions with no single right answer. And it needs a judge you can trust, whether that is a person, a second model or both, with people checking the disagreements. Three hundred examples is a start for one assistant, not a ceiling.
Control 3: A human in the loop where it matters
Output went through supervisor review. And clear cases that sit inside policy don't need a person at all. The line is drawn by what the rules can decide: anything they can't goes to a person, with the evidence already gathered.
A human is only a control if they can say no. Give reviewers the time and the context to decide, track how often they overturn the system, and treat a reviewer who approves everything as a failure in the harness, not a success. Decide in advance what happens when a reviewer gets it wrong.
We used the same split in the refunds and chargebacks digital worker at Prodaptix: inside policy goes through and is reconciled daily, outside policy or above a set amount waits for a person, and a person reviews every chargeback response. The refunds worker and the approval article cover how we set that line.
The model did the reading. Code checked the rules. A person approved the exceptions.
Matching the model to the task
Not every call needs the biggest model. Claude 3.5 Haiku handled summaries and after-call work. Claude 3.5 Sonnet handled the live calls and anything involving deal numbers, where a wrong answer costs the most.
This is a cost and quality decision, not a safety control, and it is the reason cost per conversation fell by 30–40%. That is the client's reported figure. I'm not publishing a breakdown, because the request mix and token counts are theirs.
What happened
The client reported happier customers, and lead-to-close conversion doubled with no increase in lead volume. Take that as the client's reported result, not a controlled test: other things move conversion, and I'm not claiming the model did it alone. What I can say is that the assistant gave reps better context, and the harness meant the reps trusted it enough to use it on live calls. That is where the value was.
The same pattern in miniature
The client system is private, so here is the same pattern in a small sales assistant that can send quotes. It has four tools: look up an account, get a price, send a quote and check a quote's status. Here is how it changes across four stages.
- Raw agent. The model is wired straight to the tools. A rep says "send the quote to Acme with 15% off" when the CRM has two Acme accounts, and the model guesses. It picks one, and the quote goes to the wrong customer.
- Observability first. Every state change, tool call, error and validation check is logged. This comes before any rule, because without it you can't see which rules to write or audit anything later. It also catches the quiet failure: an agent that hits its retry limit can loop in silence and burn through the token budget.
- Deterministic rules. A validation service now sits between the model's intent and the tool. If more than one account matches, execution halts and the agent has to ask, with details such as the owner, region and last deal date of each. The service also checks that the discount is above zero and within the rep's approval limit, that the price exists in the catalogue, and that the account isn't on hold. A daily send limit and a maximum number of attempts are enforced here too.
- Human confirmation. Sending splits into draft and confirm. The harness intercepts the call and puts the quote in a pending confirmation state. The rep is asked to confirm it before it goes to the customer, and only their approval releases the execution lock.
The tools and the model are the same at every stage. Only the layers around them change.
Where a harness fails
A harness is software, so it breaks like software.
| Failure | What it looks like | What guards against it |
|---|---|---|
| A new tool added without review | The agent can now write, and read-only no longer holds | Review permissions on every new tool, and a test that fails if an agent can write |
| A stale rule in the validator | It blocks valid actions, or lets wrong ones through | A named owner, reviewed whenever pricing or policy changes |
| A broken validator | It passes everything, which is worse than having none | Unit tests for every rule, and fault injection on tool failures |
| Confirmation fatigue | Reps click confirm without reading | Confirm only what matters, and show exactly what will change |
| Poisoned content | An agent follows an instruction hidden in a note or document | Treat retrieved text as data, and scope what each agent can read |
| A runaway loop | Tokens burn with nobody watching | Token budgets per call and per rep, a cap on steps, and spend alarms |
| Gaps in the logs | You can't reconstruct what happened in an incident | Alert on missing traces, and decide how long logs are kept, who can query them and how personal data in them is handled |
- Failure
- A new tool added without review
- What it looks like
- The agent can now write, and read-only no longer holds
- What guards against it
- Review permissions on every new tool, and a test that fails if an agent can write
- Failure
- A stale rule in the validator
- What it looks like
- It blocks valid actions, or lets wrong ones through
- What guards against it
- A named owner, reviewed whenever pricing or policy changes
- Failure
- A broken validator
- What it looks like
- It passes everything, which is worse than having none
- What guards against it
- Unit tests for every rule, and fault injection on tool failures
- Failure
- Confirmation fatigue
- What it looks like
- Reps click confirm without reading
- What guards against it
- Confirm only what matters, and show exactly what will change
- Failure
- Poisoned content
- What it looks like
- An agent follows an instruction hidden in a note or document
- What guards against it
- Treat retrieved text as data, and scope what each agent can read
- Failure
- A runaway loop
- What it looks like
- Tokens burn with nobody watching
- What guards against it
- Token budgets per call and per rep, a cap on steps, and spend alarms
- Failure
- Gaps in the logs
- What it looks like
- You can't reconstruct what happened in an incident
- What guards against it
- Alert on missing traces, and decide how long logs are kept, who can query them and how personal data in them is handled
What I'd tell a CTO
If you're shipping an agent, don't start by asking which model is best. Ask what the agent is allowed to do, what checks its answers before anyone sees them, how you'll notice when the business changes underneath it, where a human signs off, and what you can trace when something goes wrong. Then test the harness as hard as you test the model.
Those answers are the harness. The model proposes. The layers around it decide what actually happens.
The model is the part you can swap next quarter. The harness is what you keep.