Skip to content

RAG with citations: answers that show their sources ​

What you get: A RAG workflow whose answer carries [1]-style markers, each resolved to the chunk id, source id and score of the document it cites.

Needs: An OpenAI key as shipped; none with the demo embedder and echo · Time: 15 minutes · Nodes: 6

Retrieve runs a hybrid search over a built-in knowledge base and returns the four best chunks as numbered context. It also emits citationInstructions, a short instruction that tells the model to end each claim with the number of its source, such as [2]. The Grounded Prompt puts both into the prompt, the LLM answers, and Extract Citations finds every marker in the answer and maps it back to its chunk. As shipped, retrieval uses openai-embedder and the LLM uses gpt-4o-mini, both with the server's OPENAI_API_KEY. To try the wiring without a key, switch to the token-embedder demo embedder and the echo model.

Steps ​

  1. Download rag-with-citations-flow.json, or take it from the samples folder on GitHub.
  2. In the studio's workflow editor, import the file (see import and export).
  3. Create a knowledge base and add a document, as in the knowledge base recipe. Create it with the embedder the Retrieve node uses: openai-embedder as shipped, or token-embedder to start with no key.
  4. With no OpenAI key, use echo for the LLM node. The Provider list has no echo entry, so set the node's providerId to echo in the JSON file and import it again before you edit the nodes below.
  5. Open Retrieve and replace <your-kb-id> with the knowledge base id, in Vector Store Extra Config (JSON). If you used token-embedder, set Embedder to Token Based (Local Demo). Leave Vector Store alone: its list has no entry for the built-in knowledge base, so it may show another store, but the saved knowledge-base value stays. Save.
  6. Run it, then open the run and click Extract Citations to read the resolved citations.

How it works ​

Diagram of the RAG with Citations workflowRuns Question, then Retrieve, then Grounded Prompt, then LLM, then Extract Citations, then Output.Question (text_input)Questiontext_inputRetrieve (rag_retrieve)Retrieverag_retrieveGrounded Prompt (prompt_template)Grounded Promptprompt_templateLLM (llm_call)LLMllm_callExtract Citations (extract_citations)Extract Citationsextract_citationsOutput (output)Outputoutput
Solid arrows are the order the steps run in. Dashed lines attach a chat model, memory, tools or workers to an agent; the agent uses them while it runs.
NodeTypeRole
Questiontext_inputStep in the run
Retrieverag_retrieveStep in the run
Grounded Promptprompt_templateStep in the run
LLMllm_callStep in the run
Extract Citationsextract_citationsStep in the run
OutputoutputStep in the run

6 node types. Each type links to its config fields in the node reference.

What a run does ​

We ran the sample on a fresh install against a knowledge base holding a short helpdesk FAQ in 4 chunks (password reset, billing, data export and an error code).

As shipped, with no OpenAI key, the run stops at Retrieve: OpenAI Embedding API error: Unauthorized. The question has to be embedded before anything is searched.

With token-embedder and echo, the run succeeded. Retrieve returned all 4 chunks, the password-reset chunk first, each with its chunkId, sourceId, similarityScore and retrieval.mode: "hybrid". Echo answered with the whole prompt, so the answer contained [1] to [4] from the numbered context and [1], [2] and [3] from the citation instructions' own examples. Extract Citations found 8 markers and hasCitations: true. That shows the plumbing, not a model citing: a check that only asks "are there citations?" passes a prompt echoed back.

With a scripted stand-in model. To see real resolution without a provider key, we pointed the LLM node at a small local OpenAI-compatible server that returns a fixed answer: "Open the sign-in page, choose Forgot password and enter your work email [1]. The reset link is valid for 30 minutes [1]. If sign-in fails with E1042, your session expired: retry in a new tab [3]." Extract Citations returned 3 citations covering 2 documents (uniqueCitedDocuments: 2). Each one carries the marker, the 1-based index, the chunkId, sourceId, knowledgeBaseId, the chunk text, the score and the marker's startIndex and endIndex in the answer. An answer with no markers returned hasCitations: false and an empty list. A marker that matches no document, such as [7] with four documents, is skipped.

The Output node returns only {{answer}}, the text with its markers. The citations stay in the Extract Citations output. The studio's chat panel reads them from there and shows each marker as a link to a footnote. An app calling the workflow through the API gets only the text unless you add {{citations}} to the Output template, which renders the list as JSON.

Check that the citations hold up ​

A citation proves which chunk the model pointed at, not that the chunk says what the model claims. To check that, add an Output Guardrail node after Extract Citations with the Grounded in sources check (grounding) and On Fail left at Stop with Error. It splits the answer into claims and looks for each claim's words in the documents it cites. See grounding in sources.

We tried it with the stand-in model. The answer above passed. "The reset link is valid for 24 hours and arrives by text message [2]." failed the run with Output guardrail checks failed: grounding (block: ungrounded (output.grounding)), because document 2 is the billing chunk. An answer with no markers passed by default, since its words appear in the documents; with Every claim must cite a source (requireCitations) turned on, it failed too.

The check is lexical. It catches a claim that cites the wrong chunk or says something no chunk says. A paraphrase in new words can read as unsupported, and a changed figure inside an otherwise matching sentence can slip through. It is optional; simple workflows need none of it.

Score answers in evals ​

To test a change to the prompt, the chunking or the embedder, run fixtures through the eval framework with the RAG scorers in Knowledge bases. context_precision and context_recall are programmatic and cost nothing per fixture. faithfulness and answer_relevance call a judge model with the server's provider keys, so they need both EVAL_JUDGE_ENABLED=true and EVAL_JUDGE_ALLOW_ENV_CREDENTIALS=true. The scorers read documents and answer from the run's output. This sample's output is only result, the answer text, so the context scorers find no chunks. In the copy you evaluate, delete the Output node: the run's output is then the Extract Citations output, which carries both answer and documents.

Make it yours ​

  • Use any chat model on the LLM node. Run the grounding check above on a few answers to see how well your model cites.
  • Write your own instructions in place of the citation instructions, but keep the numbered form [1]: that is what Extract Citations resolves first. It also accepts a chunk id in brackets, such as [kbc_...].
  • Lower topK (4 in the sample) to keep the prompt short, or add a Rerank step before the prompt: see hybrid search and reranking.

Make it safer ​

These controls are optional; the recipe runs without them.

  • Retrieved chunks are outside content and can carry instructions for the model. Content rules can check each retrieved document before it reaches the model.
  • Tell readers when an answer has no citations: branch on hasCitations with an If node.
  • Cap model spend with a budget.

Further reading ​

This website (not the L2M product) uses Google Analytics, only after you accept cookies. Cookie settings