Skip to content
AI agents by job· Guide 4 of 12Intermediate9 min readUpdated

Internal docs Q&A agent on your own servers ​

An internal docs Q&A agent answers staff questions from your own handbooks, runbooks and policies, and shows which document each answer came from. The design choice that matters most is not the model: it is which documents a given person's question can reach, because whatever the retrieval step finds can end up in the answer. In L2M you start from the built-in knowledge base, a retrieval step with hybrid search, and a citation step, all running on your own server; access is controlled per project, not per document.

The job ​

InputA question in plain words, from a chat box or an internal app ("How do I get a VPN token?", "What does error E1042 mean?")
KnowledgeYour documents, split into chunks and stored with their embeddings in a vector storeVector store: A database that keeps embeddings (lists of numbers that stand for the meaning of text chunks) and returns the chunks whose embeddings are nearest to a query's.
OutputA short answer with numbered markers such as [1], each resolved to the document it came from, or a plain "the docs do not say"
NeverAn answer built from a document the asker is not meant to see, or a confident answer with no source

This is retrieval-augmented generationRAG (retrieval-augmented generation): A pattern that first retrieves documents relevant to a question and puts them in the prompt, so the model answers from that text instead of from memory alone.: find the passages that match the question, put them in the prompt, and have the model answer from them. The RAG paper by Lewis et al. framed the motivation: a model's ability "to access and precisely manipulate knowledge is still limited", and "providing provenance for their decisions and updating their world knowledge remain open research problems". Retrieval addresses both: you update the documents, not the model, and every answer can point at its source.

Agent or fixed workflow? For plain Q&A, a fixed workflow is usually better: retrieve, prompt, answer, cite. It makes one model call per question and is easy to test. Reach for an agent with tools when answering means a chain of lookups the workflow cannot predict, such as checking a ticket's status in another system before quoting the policy that applies. Workflows vs agents covers that trade-off; this page uses the fixed workflow.

The design ​

Docs Q&A workflowA Chat Trigger passes the question to RAG Retrieve, which searches a project knowledge base with hybrid search. A Prompt Template puts the numbered chunks and the question into a prompt for an LLM Call. Extract Citations maps each marker in the answer back to its chunk, using the retrieved documents. An Output Guardrail with the grounding check stops answers whose claims the cited chunks do not support, and the Output returns the answer.Questionchat_triggerRetrieve (hybrid)rag_retrieveGrounded promptprompt_templateAnswerllm_callCitationsextract_citationsGrounding checkoutput_guardrailReplyoutput
Solid lines are the order the steps run in. Retrieve feeds both the prompt and the citation step, so every marker can be resolved to a chunk.
NodeTypeWhat it does here
Questionchat_triggerTakes the message as the question. Set authMode to session (see the next section)
Retrieverag_retrievevectorStoreId: "knowledge-base", vectorStoreConfig.knowledgeBaseId set to your knowledge base, searchMode: "hybrid", topK 4 to 6
Grounded promptprompt_templatePuts in the numbered context, the citationInstructions the retrieve step emits, and the question
Answerllm_callAny chat model your deployment allows
Citationsextract_citationsResolves [N] markers to chunkId and sourceId; the chat panel shows them as footnotes
Grounding checkoutput_guardrailOptional: checks: ["grounding"], onFail: "error"

Why hybrid search. Staff ask in their own words ("the thing for remote access") and with exact strings (an error code, a form number). Dense vectors catch the paraphrase; keyword ranking catches the exact string. Hybrid searchHybrid search: Retrieval that runs a keyword search (such as BM25) and a vector search for the same query, then merges the two ranked lists into one. runs both on the built-in knowledge base and fuses the rankings (hybrid search). Two limits: the keyword side splits text into runs of a–z and 0–9, so it suits English and identifiers, and external vector stores fall back to vector-only search when you ask for hybrid.

Documents in, and kept fresh. Upload Markdown, HTML, CSV, JSON or plain text under Knowledge in the sidebar or with POST /api/knowledge-bases/:id/upload. PDF and Word files are not loaded directly: extract their text first and upload that. Give each document a stable sourceId. Uploading again does not replace a document: it appends a second copy of its chunks. On our test install, three uploads of one three-chunk FAQ under the same sourceId left nine chunks. To update a document, an admin deletes its source (DELETE /api/knowledge-bases/:id/sources/:sourceId), then you upload the new version. There is no built-in sync from a drive or wiki, so make re-uploading changed documents somebody's job.

Keeping restricted files out of answers ​

Anything in a knowledge base that a workflow can query can appear in that workflow's answers, for anyone who can run it. L2M controls this per project and knowledge base:

  • A knowledge base belongs to a project, or to nobody. A run sees the knowledge bases of its own workflow's project plus the shared ones that have no project. Any other id reads as missing. On our test install, a workflow in the default project pointed at an HR project's knowledge base: retrieval returned no documents and the run still succeeded, even when an admin started it. The same workflow saved in the HR project retrieved the salary-band chunk. So a wrong id fails quietly: test that answers come back, not just that the run is green.
  • Who can change the content. Only admins create and delete knowledge bases and delete sources. Builders with write access to a project add documents to that project's knowledge base. A shared knowledge base is read by every project's runs, so only an admin may add to it. Creating, updating and deleting a knowledge base, uploads and source deletions are recorded in the audit log.
  • Who can ask. A Chat Trigger in session mode only answers signed-in users who can read the workflow's project. Its default, when no mode is set, is public, which checks nobody; bearer checks a shared token, not a person. With PROJECT_ISOLATION=strict, users see only the projects they are members of (project access).

What L2M does not do:

  • No per-user or per-document filtering. Retrieval does not know who asked. It does not read the permissions of the system a document came from, and the built-in knowledge base has no metadata filter on the retrieve step. Everyone who can run the workflow can get any chunk of its knowledge bases.
  • So the pattern is one knowledge base per audience, in that audience's project: an all-staff handbook shared or in an all-staff project, HR policies in an HR project with an HR-only Q&A workflow, and so on. Never put a restricted file in a shared knowledge base "just for now".

Two optional guards: documents are outside content, and a page can carry instructions aimed at the model. Content rules can check retrieved chunks before they reach the model. And if the embedder or a reranker is a hosted service, the question and the chunks leave your server; data labels and destinations can block or require approval for that.

Approvals ​

Plain docs Q&A never writes, so it needs no approval per answer; one would only teach people to click through. If you turn it into an agent that can also open a ticket or change a setting through MCP tools, switch on Ask before write tools: the run pauses before each write call and shows the exact arguments.

Citations and "the docs do not say" ​

The retrieve step emits citationInstructions, a short instruction that asks the model to end each claim with the number of its source and to say plainly when the context has no answer. Extract Citations then resolves every marker to its chunk. Two things to know from our runs with a scripted stand-in model:

  • A marker that matches no retrieved chunk, such as [7] with three chunks, is skipped, and hasCitations stays true when any other marker resolved. hasCitations alone does not catch an invented source.
  • The Output Guardrail's grounding check does. It failed the run for the answer with [7], for an answer that cited chunk 3 for a claim that came from chunk 2, and for a claim no chunk supports, and passed a correct answer. The check compares words, so a paraphrase can be flagged and a changed figure inside a matching sentence can slip through (grounding in sources).

How to test it ​

Build a golden setGolden set: A curated collection of reference cases, each an input with a known-good result, that a new version of an AI workflow or agent must pass before it ships. of 20 or so real questions before you open it to staff (golden sets). Include questions it must refuse, not only ones it must answer:

CaseInputA pass
Plain lookup"How do I reset my password?"Cites the helpdesk FAQ; says the link is valid for 30 minutes
Exact string"What does E1042 mean?"Retrieves the error-code chunk (the case hybrid search is for)
Not in the docs"What is our parental leave in Spain?"Says the docs do not cover it; no citation
Wrong audience"What does band L5 pay?" asked through the all-staff workflowNo salary figures, because the HR knowledge base is out of reach
Stale answerA question whose answer changed last monthQuotes the new version only (catches duplicate chunks)

Score with contains or regex on the path answer for the facts, status, and the RAG scorers: context_recall checks that the retrieved chunks contain the expected answer's words, faithfulness asks a judge model whether the answer sticks to the chunks (RAG scorers). The RAG scorers read documents and answer from the run's output by default (contextPath and answerPath change that), so the workflow you score must return both. The design above does not: the Output Guardrail passes on only the answer, and the Output returns it as result. Extract Citations' output holds both answer and documents, so score a copy of the workflow that ends there.

A replay of a golden run answers the retrieval step from the recording, so it tests a prompt or model change on the same chunks. A change to the documents, the chunking or the embedder needs a live eval run of hand-written cases.

What it costs to run ​

Per question: one embedding of the question, one model call, and with a reranker one rerank call. The model call is the big one, and its prompt is mostly retrieved text: topK times the chunk size, plus the instructions. Halving topK or the chunk size roughly halves the input tokens. Uploads cost one embedding per chunk, each time, another reason not to upload a document twice. Cap spend with a budget on the workflow or the project; a blocking budget refuses new runs once it is used up.

Starting point, honestly ​

Three samples cover the parts; none is the whole job.

  • rag-knowledge-base-flow.json: question, retrieve from a built-in knowledge base, prompt, model. It runs with no key, on the token-embedder demo embedder (poor search quality) and the echo model, which returns the prompt. Its template asks for chunk ids while the context shows numbers, so it does not produce resolvable citations.
  • rag-with-citations-flow.json: adds hybrid search, the citation instructions and Extract Citations. As shipped it uses openai-embedder and gpt-4o-mini.
  • rag-hybrid-rerank-flow.json: retrieves 20 candidates and reranks them with a hosted reranker (Cohere as shipped, with a key).

What you add yourself: a Chat Trigger in session mode in place of the text input, the workflow and its knowledge base in the right project, a real embedder and model, the Output Guardrail, a way to keep documents current, and the golden set. The recipes have the import steps.

In the live demo, the citations workflow opens in the studio's editor so you can read its wiring. It has no recorded run, and the demo does not answer questions.

Related jobs: answering a security questionnaire is the same retrieval with a review step, and a research agent with citations adds a supervisor. A Slack assistant is a natural front door for the same knowledge base.

Further reading ​

Last updated:

This website (not the L2M product) uses Google Analytics, only after you accept cookies. Cookie settings