Retrieval (RAG)

What it is

Finding the most relevant passages for the task and handing them to the model as its sources.

Why it matters

The model answers from your documents, not from general internet knowledge, and it can cite where each statement came from.

Risk without it

Without retrieval the model guesses; with poor retrieval it confidently cites the wrong passage.

How it works

The task becomes one or more queries. Hybrid search returns a wide candidate set (say the top 40), a reranker scores each candidate against the query, and only the best few go forward. Filters keep results within the right client and timeframe.

Typical stack

• hybrid search
• Cohere Rerank / cross-encoders
• query rewriting

Pitfalls & trade-offs

• Evaluate retrieval on its own (recall@k), separately from the final answer.
• Rerankers add latency, so cap the candidate set.
• Query expansion helps vague questions and hurts precise ones.