The essentials to test before you go live

Two patterns keep coming up when AI agents go live, and they’re opposites.

Some teams move fast. The launch date has to stay the launch date, development runs over, and it’s usually the testing that gets squeezed. That’s the part that exposes what could go wrong, so you can fix it before customers find it.

Others move slowly. I’ve been in meetings of over 20 people, each signing off on their part of the business. It feels like a committee, and someone always asks, “Are we ready now?” There always seems to be one more thing, so the launch drags. Until you’re live, you’re not learning from real customers.

Both lack an agreed set of essentials: the few things you must test and prove before an agent meets a customer. Until then, you can judge how bad a failure would be, but only guess how often customers will trigger it. So anything that can hurt you in one conversation is an essential. Everything else, including how well it delivers the outcome you built it for, improves through learning and iteration once you’re live.

The essentials

Every agent’s essentials differ, but they fall into three groups: customer experience, security and legal. All three protect brand reputation, and that’s what organisations are most afraid of losing.

Customer experience is how each conversation with your agent feels to the customer.

• It sticks to the facts, and says when it doesn’t know.
• It stays polite, on brand and on topic, however rude the customer gets.
• It treats every customer fairly, whoever they are and however they say it.
• It hands over to a person when the customer asks, gets stuck or is vulnerable.

Most organisations I’ve worked with ask, “How will our agent react to a rude customer?” Even if the customer is baiting it, one rude reply is enough to get bad press. In some markets, the bigger worry is the agent talking about the authorities, politics or historical events.

Security is what your agent can reach, and what it can be talked into.

• It holds its rules when someone, or something it reads, tries to talk it out of them.
• It never leaks customer data or internal information.
• It only takes actions the customer is authorised to take.

DPD’s chatbot swore after a customer told it to “disregard any rules”. That’s a jailbreak, a form of prompt injection, which tops OWASP’s Top 10 for LLM Applications. Read it alongside OWASP’s Top 10 for Agentic Applications.

The UK’s National Cyber Security Centre warns prompt injection “may never be totally mitigated”, so don’t rely on the model to protect itself. Limit what it can reach, and have your own systems verify the customer and authorise each action.

Legal is what you can be held to, and that depends on your country, your sector and the regulators you answer to. In the UK, the Competition and Markets Authority says you’re as responsible for an AI agent as for an employee.

• It never promises or advises beyond your policies.
• It never misstates a customer’s legal rights.
• It only asks for the personal data it needs.
• It says it’s an AI, from the first message.

In the EU, AI disclosure has been legally required since 2 August 2026, unless it’s obvious.

If your agent holds up on the essentials, you’re ready to launch. Start small and build up from there.

How to test them

Run each test many times, in different words, because the same question can get different answers. The worse the harm, the more runs and the higher the pass mark.

Teams test the way they think, not the way customers behave, so be rude, switch languages and keep going. Problems often creep in during long conversations, once they pass 10 turns. Don’t let the builders mark their own homework, give security testing to specialists, and keep the evidence. Once you’re live, watch closely with an off switch ready, and retest after every change, including your model provider’s.

Before the launch date is set, agree the essentials and their pass marks with everyone in those meetings, so there’s an answer when someone asks, “Are we ready now?” The fastest launches I’ve worked on still tested the essentials. They were just quick about it.