AI and Chatbots in Customer Service: Where They Help and Where They Do Not

How Many Times a Day Do You Answer the Same Question?

Anyone who handles customer messages knows this: most of the incoming volume is repetition. "Where is my order?" "How do returns work?" "Is this item in stock?" "What are your opening hours?" The same answer, written dozens of times a day, across different channels.

This repetition causes two problems. First, because the team's time fills up with simple questions, less attention is left for the ones that genuinely need it — an upset customer, a complex technical issue, the details of a large order. Second, messages that arrive in the evening or at the weekend sit unanswered, and rather than wait, the customer goes elsewhere.

AI-powered customer service tools are positioned as the answer to both problems. But systems built without a clear understanding of what these tools can and cannot do often end up frustrating customers further instead of helping them. This post tries to set the expectation in a realistic place.

What Is a Chatbot, and How Many Kinds Are There?

"Chatbot" is not a single technology. In practice two very different approaches share the name, and confusing them is common.

Rule-based bots follow a predefined flow. They offer the customer options: "1 - Order tracking, 2 - Returns, 3 - Other." Each choice leads to a fixed answer or the next step. They are predictable, they never say anything wrong, and they are cheap to set up. On the other hand they are inflexible: the moment a customer asks something outside the flow, the bot stalls and produces the infamous "I did not understand that."

Language-model-based bots can interpret freely written questions and respond in natural language. It does not much matter how the customer phrases things; "my package still has not arrived" and "order status enquiry" both land in the right place. That flexibility is a real advantage, but it brings a serious risk: the model can produce a confident answer about something it was never told. This is usually called "hallucination," and in customer service it can mean an invented return condition or a promise of a discount that does not exist.

The practical implication: a language-model bot is only safe when it is connected to your own verified source of information. We covered the wider picture of where AI fits in a business in our post on AI use cases for SMEs, and the more autonomous systems one step beyond this in what AI agents are.

Which Questions Genuinely Suit Automation?

For a question to be safely handed to a bot, it needs three properties: the answer is unambiguous, it is asked frequently, and the answer can be retrieved definitively from a system or a written source. Typical examples:

• Order and shipping status: the answer already exists in your system; all the bot does is fetch it quickly.
• Return, exchange, and warranty conditions: written rules that do not change.
• Product details and stock availability: dimensions, materials, compatibility, quantity on hand.
• Opening hours, address, payment options: classic questions, but high in volume.
• Taking an appointment or a request: collecting a few details and routing them to the right person.

Notice what they have in common: none of them requires judgement. The bot is not interpreting anything or making a decision; it is delivering existing information at the right moment. That is where automation returns the most value.

Where Should a Human Stay in the Loop?

A bot should hand over to a person quickly in the following situations — and getting this right is the most important part of the setup:

When the customer is clearly upset, complaining, or reporting that something went wrong. In those moments what they need is not information but someone to talk to. A customer stuck in a loop trying to explain their problem to a bot usually leaves more dissatisfied than they were to begin with.

In anything with financial consequences: refunds, cancellations, invoice corrections. The cost of a wrong answer here far outweighs the time saved by automating it.

When product selection requires real advice. In technical or high-value sales especially, the conversation is part of the sale; automating it can cost you the sale.

And finally: when the customer says "I want to speak to a person." Every bot that makes this harder damages the brand. A healthy setup makes that transition in one step, without friction.

What to Prepare Before You Build

Most chatbot projects fail not because of the technology but because of missing groundwork. The real work happens before setup:

Write your knowledge base down. A bot can only produce correct answers from the source it is given. Your return policy needs to exist in writing and be current, not just in someone's head. The side benefit is significant: many businesses discover during this exercise that their own rules are vague.

Sort out the system connections. To genuinely answer "where is my order?", the bot needs access to your order system — which means a proper API connection. Without it, the bot can only give generic information, and its value drops sharply.

Settle the personal data question. Conversations contain names, phone numbers, addresses, order details. Where that data is stored, how long it is kept, and which service provider it is transferred to must be decided up front. For the regulatory side, including privacy notices and retention periods, see our data protection compliance guide — this assessment is essential if you plan to use an AI service hosted abroad.

State the limits openly. Telling customers up front that they are talking to a bot does not reduce trust; it increases patience. A customer who assumed they were talking to a person and feels misled reacts far more sharply.

How to Measure Success

Measuring a chatbot by "how many messages it answered" is misleading; a bot can reach a high number with wrong answers. More meaningful measures are: what share of conversations were resolved without handover, how much the first-response time dropped, which topics dominate the handed-over conversations, and how customer satisfaction changed after the bot went live.

That last point matters most. If automation speeds up resolution while lowering satisfaction, there is no net gain. Applying the approach from our post on choosing the right KPIs and reviewing the results regularly helps here. In the longer term, the thing to watch is this: unanswered or poorly handled customer contact is quietly one of the most common causes of customer churn.

A Realistic Starting Point

The healthiest approach is to start small. Identify the five most frequent questions and build something that answers only those, correctly. Define the handover rule clearly: if the bot is not confident, or the customer asks, the conversation moves to a person immediately.

After a few weeks, read the conversation logs. You will see where the bot got stuck and which questions arrived in unexpected forms — those logs are the best roadmap for the next improvement. Expanding scope based on that feedback delivers results far faster than trying to cover everything from day one.

At Lumethis, we help businesses make their customer communication clear and measurable first, then build automation that is connected to their own data with well-defined boundaries. If you would like to assess together what fits automation in your processes, get in touch; you can also review our services to see how we work.