Mantia Essentials assessment
Test your AI chatbot or agent before it goes live. Mantia holds realistic conversations with it to find weak points, so you know what to fix.
£250
One Mantia Essentials assessment
- Over 100 multi-turn conversations
- Your agent tested against 20 goals
- A full, evidence-backed report
*From a successful connection to your agent and the start of testing.
- Your agent 10 minutes to set up
- Mantia simulates your users 100+ conversations
- Your report within 24 hours*
20 goals, all included.
What we test
Each goal is one thing your agent must get right with a real user. For each one, Mantia holds several conversations, trying a different approach each time, and judges every one against set standards and your documents.
-
Security
7 goalsChecks that it can’t be hijacked, talked out of its rules or made to leak what it shouldn’t.
See the 7 goals
- Ignores instructions hidden in text pasted into the chat
- Keeps its instructions and settings confidential
- Declines tasks that aren’t part of its job
- Resists jailbreaks that try to give it a new identity or rules
- Resists fake claims of authority
- Keeps passwords, keys and other credentials secret
- Keeps its tools, systems and technical set-up private
-
Safety
4 goalsChecks that it never says anything that could harm or abuse the people it talks to.
See the 4 goals
- Refuses to produce hateful or harassing content
- Refuses to give instructions that could cause physical harm
- Responds safely to mentions of self-harm
- Stays respectful, even with rude or difficult customers
-
Conduct
4 goalsChecks that it treats people the way your best people would.
See the 4 goals
- Offers only the refunds, discounts or guarantees it’s allowed to
- Stays professional about your company when customers complain
- Is honest about being an AI
- Remembers what a customer has already told it
-
Truthfulness
4 goalsChecks that it sticks to the facts in your documents and holds its ground under pressure.
See the 4 goals
- Never makes up policies or terms (hallucination)
- Holds its answers and refusals when a customer pushes back
- Corrects false claims about your products, prices or policies
- Cites only real sources and links
-
Privacy
1 goalChecks that it never gives away other people’s personal details.
See the goal
- Doesn’t give away other people’s personal details
Need different goals?
We’ll build them with you in a full Mantia assessment. Talk to us
What you get
Once testing is complete, your team gets a private results page showing how your agent held up.
- Every conversation in fullSee what Mantia tried and how your agent answered, message by message.
Example results page. Larkhollow Cottages is fictional. 
- Failures highlightedEach failure we find is picked out with your agent’s exact words and the turn it happened.
Example results page. Larkhollow Cottages is fictional. 
- How serious, and whyEach failure is graded by severity, with the reason it failed and why it got that grade.
Example results page. Larkhollow Cottages is fictional. 
Getting started
Three short steps, about 10 minutes, all online.
Tell us about your agent
What it does, what it must never do, and the documents that hold the right answers, such as policies or help pages. We use them to shape the conversations and to check its answers.
Connect us to it
Give us your agent’s API endpoint and, if it needs one, a key made just for us. Then paste one example request as a cURL command, so we can see how it talks.
Pay, and we start testing
Check your answers and pay. We’ll email you when testing starts, and your results are due within 24 hours of that.
Questions
What if you can’t connect?
Full refund if we can’t connect to your agent, if you cancel before testing starts, or if your results are more than one business day late.
Will it work with our agent?
Yes, if it chats in English, in text, through an API we can reach over the internet, and answers within 30 seconds.
Do you need our live system?
No. We connect to a staging or other non-production environment, so your customers and their data are never involved.
Is it safe to give you a key?
Make a key just for us that only works in the environment you give us. Only the members of our team who work on your order can see it, and we delete it 7 days after your report. Revoke it when you have your results.
Which AI models do you use?
Mantia Essentials uses models from Anthropic and OpenAI, only on terms under which they don’t train their models on what we send. A different model, from a different provider, judges every conversation, so the one that tests your agent never marks its own work. Our sub-processors page has the details, and our Privacy Policy explains how we handle your data.
How long do you keep our data?
We delete your key 7 days after your report, and your documents and set-up answers 30 days after it. Your results page stays up for 90 days, or up to 12 months if you ask. We keep a record of the order itself for 6 years after the end of the financial year of your order. Read our Privacy Policy.
Who is it for?
Businesses only. You sign in with your work email.
When does testing start?
Once we’ve connected to your agent and your documents are ready, we email you to say testing has started, with the time your results are due, and again when they’re ready. We only start testing Monday to Thursday, UK time, so if we connect on a Friday or at the weekend, testing starts the following week.
Is this a certification?
No. It is a point-in-time test of the version we can reach, with the evidence for everything we find.
Need a full Mantia assessment?
Bespoke goals, deeper testing, regulated sectors: we build an in-depth assessment around your needs.