Skip to content

Hybrid search and reranking for better RAG ​

What you get: A RAG workflow that takes 20 candidates from keyword and vector search combined, reranks them, and answers from the best 3.

Needs: A rerank key (Cohere, Jina or Voyage) and an OpenAI key · Time: 15 minutes · Nodes: 6

Retrieve (hybrid) searches a built-in knowledge base two ways at once: BM25 keyword search and vector search, 50 candidates each. It merges the two lists with reciprocal rank fusion and keeps the top 20. Rerank sends those 20 to a hosted reranking model (Cohere as shipped), which scores each chunk against the question and keeps the best 3. The Grounded Prompt and LLM answer from those. As shipped it needs a Cohere key for the rerank and an OpenAI key for the embedder and the model. Without them you can still compare the search modes with the demo embedder.

Steps ​

  1. Download rag-hybrid-rerank-flow.json, or take it from the samples folder on GitHub.
  2. In the studio's workflow editor, import the file (see import and export).
  3. Create a knowledge base and add documents, as in the knowledge base recipe, with the embedder the Retrieve (hybrid) node uses (openai-embedder as shipped). Put its id in place of <your-kb-id> (on the node, in Vector Store Extra Config (JSON)).
  4. Store your Cohere key as a secret. The Rerank form has no secret field, so put the secret's id in place of <cohere-key-secret-id> in the JSON file and import it again. Or delete secretRef from the file and set COHERE_API_KEY on the server. For Jina or Voyage, change providerId, model and the key.
  5. Run it and open Retrieve (hybrid) and Rerank in the run to compare their order.

How it works ​

Diagram of the RAG with Hybrid Search + Reranker workflowRuns Question, then Retrieve (hybrid), then Rerank (Cohere), then Grounded Prompt, then LLM, then Output.Question (text_input)Questiontext_inputRetrieve (hybrid) (rag_retrieve)Retrieve (hybrid)rag_retrieveRerank (Cohere) (rerank)Rerank (Cohere)rerankGrounded Prompt (prompt_template)Grounded Promptprompt_templateLLM (llm_call)LLMllm_callOutput (output)Outputoutput
Solid arrows are the order the steps run in. Dashed lines attach a chat model, memory, tools or workers to an agent; the agent uses them while it runs.
NodeTypeRole
Questiontext_inputStep in the run
Retrieve (hybrid)rag_retrieveStep in the run
Rerank (Cohere)rerankStep in the run
Grounded Promptprompt_templateStep in the run
LLMllm_callStep in the run
OutputoutputStep in the run

6 node types. Each type links to its config fields in the node reference.

Why hybrid ​

Vector search matches meaning: "I forgot my login" finds a chunk about resetting a password. It is weaker on exact strings such as error codes, product ids and rare names. Keyword search is the reverse. Hybrid runs both and merges the two ranked lists with reciprocal rank fusion (RRF): each list gives a chunk 1 / (k + rank), and the sums decide the order. A chunk near the top of both lists wins. The sample uses k = 60, the constant from the paper that introduced RRF, which found it near-optimal but not critical.

Hybrid and bm25 modes work only with the built-in knowledge base. With an external vector store the node falls back to vector search without an error. If keyword search finds no chunk at all, hybrid returns the vector results unchanged.

What a run does ​

We ran the sample on a fresh install with no provider keys, against a knowledge base holding a short helpdesk FAQ in 4 chunks, one of them about error code E1042.

As shipped, the run stops at the first step that needs a key. With openai-embedder and no key, that is Retrieve: OpenAI Embedding API error: Unauthorized. With the token-embedder demo embedder, retrieval succeeds and Rerank stops: Reranker "cohere" requires an API key — pass via secretRef or set COHERE_API_KEY. Note that while the node holds a secretRef, even a placeholder one, it uses only that secret and never reads COHERE_API_KEY. Delete the secretRef to use the environment key.

The search modes, compared. We removed the Rerank node and ran the retrieve step in each mode with topK: 3:

Questionvectorbm25hybrid
What does E1042 mean?E1042 chunk first, then billing, then resetthe E1042 chunk onlyE1042 chunk first (both rankers), then billing, then reset
How do I reset my password?reset chunk firstthe reset chunk onlyreset chunk first (both rankers)

Keyword search returns only chunks that share a word with the question; vector search always fills topK. In hybrid mode each chunk records which rankers found it (retrieval.rankers) and its rrfScore: 0.0328 for a chunk ranked first by both (2/61), 0.0161 and 0.0159 for chunks only vector search found (1/62, 1/63). With 4 small chunks the order barely moves. Hybrid matters more on larger collections, where an exact code can sit far down a vector ranking; measure it on yours (below).

The rerank step was not observed, as we had no rerank key. From the code: Rerank sends the question and the text of each upstream chunk to the provider (Cohere's top_n is topN), keeps the order the provider returns, and adds rerankerScore and reranker: { providerId, model } to each chunk's metadata. It returns documents, a numbered context and the same citationInstructions as Retrieve. With no upstream chunks it returns an empty list and makes no call. A provider error fails the run with the provider's status, such as Cohere rerank 401: ....

Measure the difference ​

Do not take hybrid or a reranker on faith: measure them on your own questions. Build a few fixtures (a question and the answer you expect), then score each variant with context_precision (how much of what was retrieved is relevant) and context_recall (how much of the expected answer the retrieved text covers). Both are programmatic and free per fixture. They read documents from the run's output, and this sample's output is only the answer text, so evaluate a copy that ends at the step you are measuring, Retrieve (hybrid) or Rerank. See RAG-specific eval scorers and golden sets for running them on every change.

Make it yours ​

  • Tune candidatesPerRanker, bm25Weight, vectorWeight and rrfK in the retrieve node's config in the JSON file, then import it again; the node's form has no fields for them, nor for searchMode. See hybrid search.
  • Swap the reranker: jina (default model jina-reranker-v2-base-multilingual) or voyage (rerank-2), with JINA_API_KEY or VOYAGE_API_KEY or a secret. Change model too, or clear it to use the default: the sample's rerank-english-v3.0 is a Cohere model and is sent as is.
  • The prompt asks the model to "Cite chunkIds inline", but the context numbers chunks [1], [2]. Put {{citationInstructions}} in the prompt instead and add an Extract Citations step: see RAG with citations.

Make it safer ​

These controls are optional; the recipe runs without them.

  • A reranker is one more service that receives your documents' text. Keep it in mind when you choose which documents to index.
  • Retrieved chunks can carry instructions for the model: content rules can check them before they reach the model.
  • The rerank provider bills you directly, and L2M budgets do not count rerank calls. Cap the model side with a budget.

Further reading ​

This website (not the L2M product) uses Google Analytics, only after you accept cookies. Cookie settings