How Do You Make an AI Voice Agent Answer Only From Your Own Documents?

Short answer: give the agent a knowledge base built from your own material — crawl your website, import a sitemap, paste text, or upload txt, md, csv and PDF files with a text layer — and write into its instructions that it answers only from those documents. Switch off web search for that agent, and turn on hand-over for questions it can't answer, so a gap reaches a person instead of a guess. Then read the list of questions it couldn't answer and add what is missing.

Why it matters on a phone call

On a call, a wrong answer sounds just as confident as a right one, and there is no page for the caller to check. If the agent quotes a refund window, a fee or a document list, it has to be yours — the one on your website today, not a plausible one.

How to set it up, step by step

1. Gather your sources. Crawl your website one to three levels deep, or import its sitemap to pick up every page. Paste policies that aren't online, and upload txt, md, csv or PDF files.
2. Check your PDFs have text. Only PDFs with a text layer can be read — a scanned page is a picture, and there is no OCR. Word files, Google Docs and Notion pages aren't connected directly; save them as PDF with text, or paste the text in.
3. Put prices and lists in a CSV. CSV files are split by row, so a price never gets separated from the product it belongs to.
4. Write the rule into the instructions. Tell the agent to answer questions about your business only from the knowledge base and to say plainly when it doesn't have the answer.
5. Switch off what you don't want. Turn off the built-in web search for this agent, and add only the tools it needs — tools are off until you allow each one.
6. Turn on hand-over for unanswered questions. When the documents don't cover a question, the agent transfers the call or sends the conversation to your team.

Keep it accurate after launch

• Read the unanswered questions. Every search that found nothing is recorded, and the agent's analytics list those questions — the pages you need to write next.
• Replace documents when things change. Delete the old version and add the new one, so the agent never has two answers to choose between.
• Check transcripts. Every call has a transcript, so you can see exactly what the agent told each caller.

An honest limit

A language model can always reach beyond what it is given; no single setting makes that impossible. The instructions, the absence of web search, the hand-over switch and regular transcript checks are what keep the agent on your documents — and they work best together.

One knowledge base, every language and channel

Your documents can be in one language: the agent answers in the caller's, across all 22 scheduled Indian languages, and the same knowledge base serves calls, WhatsApp and website chat. Customer data is stored in India (Mumbai).

What you need to start

Your website address and the documents your team already uses to answer customers. The knowledge base page covers sources and indexing, and Indian languages lists what the agent speaks.