Why Your Chatbot Reads Your Knowledge Base Differently Than Your Customers Do

When someone reads an article in a knowledge base, they get more information than just the text itself. They know which page they came from. They can see the menu, and perhaps photos and screenshots. Anything that isn’t explicitly written down, they can often figure out from the context.

A chatbot can’t do that.

Most AI chatbots use retrieval-augmented generation (RAG). Whenever someone asks a question, the bot searches the knowledge base for the pieces of text that best match the query and uses them to formulate an answer. Those pieces of text are often individual paragraphs. The bot usually can’t see the surrounding context, such as the menu or information shown in an image.

This becomes particularly noticeable with questions that require step-by-step instructions. To illustrate this, let’s take an example almost everyone is familiar with: the Wi-Fi router. Almost every household has one, and most people have wanted to change their Wi-Fi password at some point.

Changing a Wi-Fi password can easily involve five or six steps: logging in to the router, finding the right menu, entering a new password and saving the changes. The exact steps may depend on the router model. One model might use a different login address or menu structure than another, while some routers are managed through an app.

Imagine an internet provider has a single knowledge base article explaining the steps for the router it currently supplies as standard. The router model is clear from the context surrounding the article. It’s listed under the category for that particular router, the model name appears at the top of the page, and there are photos of the device. But the model name isn’t mentioned in the instructions themselves. For customers, that’s not necessarily a problem. They can tell at a glance whether the instructions apply to their router.

A chatbot usually doesn’t get that surrounding context. It sees the instructions separately from the category, page title and images. Suppose a customer asks: “How do I change my Wi-Fi password?” The bot correctly asks a follow-up question: “Which router are you using?” The customer says they have an older model. The bot then searches for instructions for that particular model but might not find anything, because the model name doesn’t appear anywhere in the actual instructions.

At that point, one of two things may happen. The bot either provides the instructions for the standard router as though they apply to every router, or it says it doesn’t know the answer.

With step-by-step instructions, this can be particularly frustrating. If even one step is incorrect, the customer may not be able to continue. They might look for a menu that has a different name on their router or try to log in using an address that doesn’t apply to their model. The chatbot can only work with the text it receives. The problem is that the text doesn’t say which router the instructions are actually for.

The router is just one example. You’ll find the same pattern in many knowledge bases. Consider terms and conditions that apply exclusively to business customers, while that distinction is only indicated by the article’s category. Or a paragraph that starts with “as described above”, when the chatbot may never receive the preceding section at all.

In each case, the information is obvious to a human reader but missing or unclear to the chatbot.

A common first response is to configure stricter guardrails for the chatbot. Guardrails are rules that determine what the bot is and isn’t allowed to say. For example, you could configure the bot to answer only when its source material is an exact match for the customer’s question. If there’s any doubt, the bot refers the customer to a support agent.

That helps prevent incorrect answers. This is particularly valuable when dealing with information about contracts, costs or cancellation periods. The downside is that the bot will also reject many questions it could have answered perfectly well. Customers will see “Unfortunately, I can’t help you with that” more often, while the number of conversations transferred to your customer service team increases.

That can quickly undermine much of the value you hoped to gain from your chatbot. And importantly, the gap in your knowledge base still exists.

Another quick solution is to adjust the prompt. The prompt contains the instructions that determine how the chatbot behaves. You could add specific knowledge to it, for example: “The instructions for changing the Wi-Fi password only apply to the new router. If the customer has an older model, refer them to customer service.”

This works immediately. The problem is that your knowledge now exists in two places. When someone updates the knowledge base, the prompt needs to be updated as well. When a new router is introduced, the instruction in the prompt may suddenly become outdated.

After a few months, the prompt can contain dozens of these exceptions. Nobody remembers exactly why they’re there, and sometimes they even contradict each other. On top of that, the person managing the prompt is often not the same person who manages the knowledge base, making it easy for changes to become disconnected.

A useful rule of thumb is: use the prompt for behaviour and the knowledge base for content. Use the prompt to define how the bot communicates, when it should ask a follow-up question and when it should transfer the conversation to a human agent. Facts, instructions and terms and conditions belong in the knowledge base.

The more sustainable solution is to structure your knowledge base so that a chatbot can work with it effectively. That doesn’t mean rewriting everything from scratch. It does require looking at your content differently. Here are a few principles to follow:

1. Make context explicit. State clearly in both the title and the text who the information is intended for and which product or device it applies to. “Change your Wi-Fi password on the standard router supplied from 2024 onwards” gives a bot much more information than simply “Change your Wi-Fi password.”
2. Write paragraphs that can stand on their own. Repeat the subject where necessary and avoid references to other parts of the page.
3. Turn information from images and PDFs into text. A picture can be useful to a reader, but a bot often can’t interpret the information it contains.
4. Use consistent terminology. Choose between terms such as “router” and “modem”, and use your chosen term consistently.
5. Add labels. Metadata such as product type or customer segment allows the bot to search more specifically within the right articles after asking a follow-up question.
6. Clean up regularly. Remove, merge or update duplicate and outdated articles.

Your customers benefit from this too. Clear and comprehensive articles can also perform better in Google and AI search engines such as ChatGPT and Perplexity.

Knowledge bases grow over the years. A new article gets added while the old version remains online, or two colleagues independently write about the same subject. For customers, that’s usually not a major issue. They can see the date, title or section where the article appears and decide which version applies to them.

A chatbot works differently. When someone asks a question, the bot retrieves the pieces of text that most closely resemble the query. If two articles in the knowledge base say almost the same thing, both may be retrieved. The bot doesn’t necessarily know which article should take precedence. It might choose one, sometimes the outdated one, or combine information from both into a single answer.

That last scenario can quickly cause problems with step-by-step instructions. Imagine there’s one article explaining how to change the Wi-Fi password on the new router and an older article covering the previous model. The instructions are similar, but the menus and login addresses are different. The bot might take steps one to three from one article and step four from the other. The resulting answer looks perfectly logical but is still incorrect.

The bot may also use article A one time and article B the next time it receives the same question, resulting in inconsistent answers for customers.

There are several ways to prevent this:

1. Keep one article per topic. Merge articles that largely overlap into a single, up-to-date version.
2. Make differences explicit. If two articles genuinely cover different situations, such as two router models, clearly state that difference in both the title and the opening sentence.
3. Archive outdated articles. Remove them from the chatbot’s source material, even if customers still need to be able to find them elsewhere.
4. Assign an owner to every article. Make it clear who is responsible for maintaining it and who decides whether a new article is needed.

Whatever approach you choose, testing is essential. Use real customer questions from your contact centre. Customers phrase things differently than you do, and that’s exactly where you’ll discover the gaps.

You can use the results of these tests to improve two things. First, the chatbot itself. Train it to better understand the way customers phrase their questions. Think of conversational language and synonyms, such as customers saying “Wi-Fi code” when your knowledge base uses “Wi-Fi password”. You can also train the bot to recognise when it should ask a follow-up question and when it should transfer the conversation to a human agent. If your chatbot works with intents, you can add new example phrases to those intents as well.

Second, you can train your knowledge base. There are two ways to do this. The first is to add information. Every question the chatbot struggles with reveals information that is either missing or insufficiently clear. You can fill those gaps directly in the relevant articles.

The second is to link questions to an article. Many knowledge base systems allow you to add example questions to individual articles. These represent the different ways customers might ask about the same subject, such as “How do I change my Wi-Fi code?”, “I want a new password for my Wi-Fi.” or “Where can I change my network key?”

This helps the chatbot identify more quickly that a customer’s question matches that particular article. It can also increase the confidence score: the percentage indicating how confident the bot is that it has found the correct answer. If you’re using guardrails with a minimum confidence threshold, more questions can then exceed that threshold. As a result, the bot can answer more questions without you having to lower the threshold.

We see the best results when the knowledge base forms the foundation. Use guardrails as a safety net for sensitive subjects, and keep the prompt as free as possible from individual facts and exceptions.

Start small: take your twenty most frequently asked questions, check whether the answers can be understood without any surrounding context, and update them where necessary. Want to discuss how to make your knowledge base work better for both customers and AI? Contact us!