A brass key lies on a paper card traced with the outline of the original key. One notch near the tip sits past the line and glows mint green.

AI agent analysis

Conversational AI accuracy: check replies against your rules

Conversational AI accuracy means every reply matches your own rules. How to check your AI agent's answers against them, with example chats.

Parimal Savaj

5 min read

Share

Say a customer sends a screenshot. Your AI agent, the software that chats with customers for you, said a worn jacket could go back. Your policy says unworn only. Conversational AI accuracy means every reply matches your own rules. Conversational AI is another name for an AI agent. Check it by comparing real replies with a written right answer.

A wrong reply can read just like a right one, so nobody notices until a customer acts on it.

Key takeaways

  • Conversational AI accuracy means every reply matches what your business actually says. You need the right answer written down to check it.
  • Check four kinds of wrong answer: wrong rule, out of date, made up, and right fact for the wrong customer.
  • When a reply is wrong, find where it came from. Sometimes the AI agent repeated a page of yours that is wrong.
  • An accuracy check is only as right as your written answer. A wrong page passes a wrong reply.

What is conversational AI accuracy?

It means your AI agent's replies match the right answer for your business. The hard part is the words "right answer".

The U.S. National Institute of Standards and Technology (NIST) gives this definition in its guide on AI risk. Accuracy is the "closeness of results of observations, computations, or estimates to the true values or the values accepted as being true". In plain words, you need something true to compare with.

For your business, that is written down somewhere. Your returns policy, your price list, your delivery terms, your help pages. If a rule lives only in someone's head, you cannot check a reply against it.

How accurate are AI agents?

There is no general figure that tells you how accurate yours is. A score for AI models in general says nothing about your returns policy. What you can find out is whether your AI agent gets your answers right.

Do not judge replies by how sure they sound. A wrong answer can sound as calm and polite as a right one.

A wrong yes about a refund is money you pay back, or a customer you argue with. The UK Financial Conduct Authority (FCA) says firms using AI-supported tools "remain responsible for consumer outcomes". That paper is about UK financial firms, so read it as a principle, not a rule for your business. If you bought your AI agent from a vendor, the reply still went out in your name.

What should you check for accuracy?

Start with four kinds of wrong answer. Each needs something different to compare with. The examples are invented.

Kind of wrong answer Compare it with Example reply
Wrong rule Your written policy "Yes, worn items can be returned." Policy: unworn only
Out of date The current version, with the date it changed "Shipping is free on every order." The offer ended last month
Made up Your pages. If none says it, ask where it came from "Yes, we match any store's price." You have no price match
Right fact, wrong customer What the customer said earlier in the chat "Delivery is next day." The customer said they live abroad

The last row is easy to miss. The reply is true, just not for this customer.

How do you run a conversational AI accuracy test by hand?

This is how to test AI agents for accuracy with no special tools. The chats below are examples, not customer data.

  1. Pick the topics where a wrong answer costs money, and write the right answer for each. Add where it comes from. "Cancellations: the deposit is not refunded. Source: booking terms." Customer: "Do I get my deposit back if I cancel?" AI agent: "Yes, deposits are always refunded." Your line says no, so the reply fails.
  2. Find real chats on each topic. Search last week's logs for words customers use, such as "refund". Customer: "how do I get a refund for a late order" AI agent: "Your refund goes back to your card." If your terms give store credit, that reply fails too.
  3. Read the whole chat, not only the last reply. Customer: "I'm in Canada. How fast is delivery?" AI agent: "Delivery is next day." Next day is true at home. For this customer it is wrong.
  4. Find where each wrong reply came from. Customer: "Is shipping free?" AI agent: "Shipping is free on every order." Then you find the old offer page still live. The AI agent repeated it, so take the page down.

That last step matters. Sometimes the fix is a page of yours, not the AI agent.

Before launch, try hard test chats first. To sort a week of failed chats by reason, see how to evaluate conversational AI performance.

How Metricsense does this

Metricsense does not write your right answers. It reads the chats and runs your checks on them.

  1. There are more refund chats than anyone can read. You upload the chat logs from your AI agent. Metricsense reads each conversation on both sides: what the customer asked and what the AI agent answered. Both sides matter here. A right fact for the wrong customer only shows when you read the question too.
  2. Your booking terms are a page, not a test. You turn each right answer into a check, written in plain English: "Did the AI agent say the deposit is refunded on cancellation? Our terms say it is not." There is no tagging and no model training. Metricsense runs the checks you write.
  3. Your vendor wants the exact words, not a summary. Every finding links to the exact message behind it. You can show them what the customer asked and what the AI agent said.
  4. Some flagged replies turn out to be fine. A flag is not a verdict. The policy may have changed, or the check may be worded too tightly. The person who owns that policy reads the message and decides the fix: the AI agent, the page it repeated, or the check.

Illustrative data, not actual customer data

  • The customer wrote: "Can I bring my dog when I stay?"
  • The AI agent replied: "Yes, pets are welcome in all our rooms."
  • The check it failed: "Did the AI agent say pets are allowed? Our policy allows assistance dogs only."

What you learn from this: the AI agent made a promise that none of your pages make. Someone checks what it was told about pets, then decides the fix.

What stays with a person?

Writing the right answers is your job, not the software's. Give each topic an owner. When a policy changes, that person updates the right answer the same day.

A person also decides what each flagged reply means. NIST notes that deployed AI systems "are often assessed by ongoing testing or monitoring that confirms a system is performing as intended". Somebody has to say what "as intended" means for your business.

What can an accuracy check not tell you?

It is only as right as your written answer. If your help page is wrong, a reply that repeats it passes the check. So does a reply on a topic you never wrote an answer for.

Passed checks are not the same as happy customers. The FCA says collecting data "will not, by itself, show whether customers are receiving good outcomes". That paper is also about UK financial firms. An accurate reply to the wrong question still leaves the customer stuck.

We do not have a neat answer for rules nobody wrote down. Start by writing those down.

Want to see this on your own chats? A pilot reads around 50 of your conversations, with a readout inside a week. Book a free pilot on your calls or chats.

Share this article

Back to the blog