A generative chatbot can produce persuasive answers that conflict with your prices, delivery times, or contracts. This article explains how source control, fallback behavior, human handoff, and secure logs reduce the risk.
Most people see a generative chatbot as a quick way to reduce pressure on customer support, but the real challenge is not making it respond fluently. It is preventing it from promising things the company has never approved. Imagine a visitor asking about a notice period and receiving a confident answer assembled from an outdated FAQ and a current offer. The answer sounds plausible, but the stated term does not exist. Behind searches for AI chatbot for websites, therefore, lies a more important question: what information should the model be allowed to provide at all?
An incorrect answer about opening hours is irritating. A fabricated price, delivery date, or refund promise, however, can create additional manual work, complicate a sale, and make the customer doubt everything else the company communicates. The solution is not a longer instruction for the model, but an architecture in which facts are controlled, business-critical values are locked, and uncertain questions are handed over.
AI chatbot for websites: a system prompt is not enough
A system prompt tells the language model how it should behave, but it does not create a secure boundary around what the model knows. The instruction “never make anything up” may reduce certain errors, but the model is still designed to generate a probable answer. If it can search the entire website, old PDF files, and its general knowledge, it may combine accurate details into an incorrect conclusion.
A more robust solution uses RAG, or retrieval-augmented generation. In this context, it means that the chatbot first retrieves relevant passages from a restricted knowledge base and then formulates its answer. The business benefit is straightforward: the company can control which information is allowed to influence customer-facing responses. The knowledge base should contain only approved documents, and each source should be labelled with details such as document owner, product, market, validity status, and most recent approval. Drafts and outdated versions are filtered out before anything is sent to the model.
The first thing we examine is whether every factual claim can be linked to an actual retrieved passage. Visitors can then be shown the document title or a link, while internal stakeholders can review the exact version used. If the search fails to find sufficiently relevant support, a separate stop rule should block the answer. The model must not fill the gap with general knowledge simply to keep the conversation going.
Prices and terms should be data, not free text
Explanatory content and customer promises require different treatment. A language model can summarise how a service works, but prices, discounts, contract periods, delivery areas, warranties, and cancellation terms should be retrieved from a controlled business system, API, or rules registry. These sources provide a defined value and a clear validity status, rather than wording that the model interprets based on probability.
The model can identify which product the customer means and extract details such as customer type, variant, and currency. The price itself, however, should be retrieved by a separate system and displayed using a locked response template. If a required field is missing, the chatbot should ask a follow-up question or hand the case over. It must not select the most common option, calculate its own discount, or complete an incomplete term.
This separation also reduces the cost of making changes. When an offer is updated, the business only needs to correct the value in the source system rather than find every piece of wording the model may have encountered. For a business AI chatbot, this is the crucial difference between content that sounds credible and information that has actually been approved.
Uncertain questions should reach a human before they become promises
When should the chatbot decline to answer? The decision should be based on observable signals, not on the language model claiming that it feels uncertain. The relevance of search results, the presence of an approved source, the category of the question, required data fields, and any contradictions provide a much more reliable basis for the decision.
Complaints, requests for special prices, individual commitments, and sensitive personal matters can be routed directly to a human representative. The same applies when sources provide conflicting information, contractual details are missing, or the customer and the bot have repeatedly misunderstood each other. These rules make the handoff predictable and prevent the model from trying to be helpful in situations where a definitive answer requires human accountability.
A good handoff includes the question, relevant conversational context, and the sources the bot attempted to use. The customer does not have to start again, while the representative can see why the answer was blocked. The standard response should explain what the chatbot cannot confirm, avoid provisional promises, and provide a specific contact option. Only the information required for the case should be transferred.
Measure whether the chatbot is controllable, not merely friendly
A friendly conversation can conceal poor fact-checking. You should therefore monitor source coverage, meaning how often business-critical answers are supported by an approved source, and review how often the bot attempts to answer despite lacking that support. Other useful metrics include the use of outdated sources, conflicting documents, correct handoffs, and recurring errors after an issue should have been resolved.
The test suite should include ambiguous product names, incorrect assumptions, claims about verbal promises, and questions designed to pressure the chatbot into offering discounts or exceptions. Check both that the bot blocks risky answers and that it does not unnecessarily block simple questions. The goal is not the highest possible automation rate. A better business metric is the proportion of automated answers that can be verified and how quickly a detected error can be traced to the correct source, rule, or data version.
Logging must also be designed with restraint. If you are assessing chatbot and GDPR issues, identifying information should be limited, access controlled, and retention aligned with a clearly defined business need. The EU AI Act also makes transparency around AI use relevant, but this article does not constitute legal advice. For legal guidance on the EU AI Act and chatbots, use official information from EUR-Lex, and for data protection, consult Swedish guidance from the Swedish Authority for Privacy Protection, IMY.
Criteria that reduce the risk of fabricated customer promises
Do not assess a chatbot solely by how naturally it responds. What matters is whether its answers can be controlled, verified, and handed over to a human when the supporting information is insufficient.
Require answers from an approved knowledge base
The chatbot should retrieve facts from restricted, quality-assured sources. There should be clear document ownership, version management, and a process for updating content when offers or working practices change.
Signal: A good supplier can show which sources were used for an answer and demonstrate what happens when information is missing.
Retrieve prices and terms as structured data
Prices, discounts, delivery times, and other customer promises should come from a controlled system or API. The model may explain the result, but it should not interpret or fill in missing values itself.
Signal: Be cautious if the language model is allowed to generate business-critical information freely from documents, older conversations, or probable assumptions.
Test how the chatbot handles uncertainty
Try ambiguous questions, incorrect assumptions, and requests for exceptions. A safe chatbot must be able to decline to answer when the supporting information is contradictory, outdated, or too weak.
Signal: The chatbot should say that verified support is unavailable rather than provide a definitive answer that cannot be substantiated.
Define clear rules for human handoff
Decide which topics must always be escalated and who will receive them. Relevant context should be included without exposing more customer information than necessary.
Signal: The supplier can demonstrate specific handoff rules, identify the responsible recipient, and show what the customer is told when a human needs to take over.
Ensure traceability and continuous quality control
It must be possible to identify the supporting information, data version, and rules behind a business-critical answer. This makes it possible to prevent the same error from recurring.
Signal: Responsible staff can review answers and sources without logging resulting in unnecessary retention or overly broad access to sensitive information.
Start here: stop the chatbot from inventing customer promises
-
Immediately restrict what the chatbot can promise
Block statements about price reductions, delivery dates, warranties, refunds, and special terms when an approved source is unavailable. Add clear instructions in tools such as the OpenAI API, Microsoft Copilot Studio, or the chatbot’s administration platform, but supplement them with technical stop rules. Uncertainty should lead to a restricted response or escalation, not a well-written promise.
-
Create a single approved knowledge source
Collect current prices, product limitations, delivery terms, and standard responses in a version-managed environment such as SharePoint, Notion, or your CMS. Appoint a named content owner and remove old documents and drafts from the search index. This creates one clear place where conflicting information can be corrected.
-
Require the chatbot to answer from verified information
Connect the knowledge source using tools such as Azure AI Search, OpenAI File Search, or a RAG solution with Pinecone. Require source support for answers about customer terms and display the document title or link where appropriate. When relevant material is unavailable, response generation should be stopped so the knowledge gap becomes visible and can be addressed.
-
Build a clear human handoff process
Send questions with uncertain supporting information or requests for non-standard terms to Zendesk, Intercom, or HubSpot, together with the conversation and the sources used. Only authorised employees should confirm individual prices, schedules, and exceptions. The customer receives faster assistance because the representative already has the context.
-
Test high-risk questions and review answers continuously
Build a test suite in Google Sheets, Airtable, or promptfoo containing questions about discounts, warranties, exact delivery promises, and exceptions. Log relevant technical events in tools such as Langfuse or Azure Application Insights and review anomalies regularly. Restrict access to identifying information and use recurring error patterns to improve sources, rules, and handoffs.
For anyone who began by searching for “AI chatbot for websites,” the next step is not to choose the most eloquent demo, but to test source tracing, blocking, and human handoff. Do not launch until the chatbot can consistently decline to answer, refer to approved supporting information, and hand sensitive questions over to a human. Start with a limited use case and expand only after documented testing. An experienced team such as FLAR AB would approach the work by starting with the narrowest useful function and building the controls before extending its reach.
