Gains:
- Implementing hybrid search that combines top-k selection and semantic and keyword search
- Increasing the accuracy of the first search with re-ranking
- Making difficult questions more searchable with techniques such as query transformation and HyDE
You shredded the documents nicely and put them into the vector database. Now the real work: fetching the right pieces when the user asks a question. This is called retrieval and is the backbone of RAG quality. Remember the phrase "Retrieval quality = RAG quality"; This unit is exactly the art of improving that quality. We will cover practical techniques from top-k selection to hybrid search, from re-ranking to query transformation.
Top-k: How Many Pieces Will We Bring?
The most basic setting: how many pieces (k) to return from the search. If you bring less (k=1) there is a high risk of missing the correct piece; If you add too much (k=20), it will add noise to the model and the token cost will increase.
Distinguish between two concepts. Recall: the rate at which the correct piece is among those retrieved. Precision: the rate at which what is retrieved is relevant. Increasing k increases recall but decreases precision. The goal is to balance the two.
top-k
Impact
suitable situation
1-3
High precision, risk of miss
Simple, one-answer questions
4-8
Balance; most scenarios
General corporate RAG
10-20
Higher recall, noise increases
With re-ranking (below)
Tip: Common and powerful pattern: fetch wide (k=20), then narrow with re-ranking (top 4). This way you won't miss anything and you give clean context to the model.
Hybrid Search: The Power of Two Searches
Semantic (vector) search captures the meaning but misses things: full product code ("XR-4471"), rare abbreviation, proper name, version number. For these exact-match tasks, old-school keyword searching—especially the classic algorithm called BM25—is very good.
Hybrid search combines the two: both semantic and keyword searches work, combining the results. Thus, in a question such as "Warranty period of the
Merging is usually done with RRF (Reciprocal Rank Fusion): the track that is higher in both lists comes forward.
# Hybrid retrieval (conceptual)sem = vector_search(question, k=20) # meaningkey = bm25_search(question, k=20) # fullwordresult = rrf_merge(sem, key)[:8] # fuse the two, top 8
Re-ranking: Second and Smarter Pass
The first call can be quick but rude. Re-ranking scores the candidates returned by the first search one by one with a more powerful model (usually a cross-encoder — a model that reads the question and the passage together and scores the relevance) and moves the most relevant ones to the top.
The logic is this: the first search assigns a large number of candidates for recall (20 candidates), the re-ranker selects the best 4 for precision. This two-stage approach is much more accurate than a single-stage search. It costs some delay and additional processing; The gain is a significant increase in quality.
# Two-stage retrieval (conceptual)candidates = hybrid_search(question, k=20) # broad, quickscore = reranker.score(question, candidates) # relevance score for each candidate context = rated.rank()[:4] # top 4
Transforming the Query
Sometimes the problem is not in the search, but in the question itself. If the user asks “what about remote workers?” ', this cannot be sought alone (it is not clear what is for remote workers). Here are the query transformation techniques:
- Rewrite: Use conversation history to make the question standalone: "What are remote workers entitled to annual leave?"
- Multi-query: Generate 3 different expressions of the same question, search with all of them, combine the results. Different words find different pieces.
- HyDE (Hypothetical Document Embeddings): First print a "possible answer" to the model, then embed and search for that imaginary answer. The imaginary answer is sometimes found better because it is closer in words to the real document.
# Multi-query (conceptual)variants = model.uret("Express this question in 3 different ways: " + question)tum_sonuc = []for v in variants: all_sonuc += vector_intermediate(v, k=6)context = singular_and_order(tum_result)[:6]
Weak Retrieval / Strong Retrieval
Weak (single stage, semantics only, constant k=3):
result = vector_search(question, k=3)# Problem: product code is missed, cannot enter correct part 3 in difficult questions.
Powerful (hybrid + widek + re-ranking + multi-query if necessary):
candidates = hybrid_search(rewrite(question, past), k=20)context = reranker.score(question, candidates).first(4)# Both exact match and meaning are captured; It goes to the best 4 models.
Three Mini Cases
Case 1 — Product code escaped. A pure semantic search on a support team couldn't find "RTX-9080" verbatim in the question "RTX-9080 driver error"; He was bringing other products with similar names. When hybrid search was added, exact code matching occurred and the rate of returning correct articles increased from 58% to 92%.
Case 2 — Re-ranking difference. A paralegal was working with k=5 but the correct item is 7-10 most of the time. He was staying in the ranks. When k=20 + re-ranking was introduced, the rate of the correct article being in the top 3 increased from 61% to 94%; Latency increased by only 300ms — an acceptable compromise.
Case 3 — Follow-up question without context. In an HR assistant, the user asks “what about part-timers?” When I asked, the system brought irrelevant parts. By rewriting the query (pulling the "annual leave" context from the past and adding "Part-time employees are entitled to annual leave"), the correct answer rate increased from 40% to 86%.
Common mistakes
- Relying solely on semantic search: Product code, abbreviation, proper name are missed; Add hybrid.
- Keeping k too small: The correct piece cannot enter the list; make it wide and narrow it with re-rank.
- Never thinking about re-ranking: First search is rude; The second smart pass significantly improves the quality.
- Searching for follow-up questions as is: A question without context searches for meaningless; rewrite.
- Blindly increasing k (without re-rank): The model is filled with noise, the response is distorted and the cost increases.
Beware: Each technique adds cost and delay. Multi-query means 3-fold search, re-ranking means additional model call. Start with simple hybrid + reasonable k first; Measure and add heavy techniques where really needed. Adding without measuring.
In summary
- Top-k is the balance between recall and precision; Generally 4-8 is a good start.
- Hybrid search combines semantic search with keyword (BM25) search; Recovers exact matches.
- Re-ranking re-scores the candidates from the broad initial search with a robust model and selects the best ones.
- Query transformation (rewriting, multi-query, HyDE) makes difficult and context-free questions searchable.
- Strong pattern: broad fetch → merge with hybrid → narrow with re-rank; but add each technique by measuring it.
Application task
Prepare 6 difficult questions with data from previous units: at least 2 requiring exact matches (product code, abbreviation, date), 2 semantic (same topic with different words), 2 follow-up questions without context. (1) First think of each question as a pure semantic search and manually mark which parts will come up. (2) Re-evaluate the same questions with hybrid and re-ranking added. (3) Rewrite follow-up questions without context. Note in a tabular form which technique makes a difference in which question type.
checklist
- [ ] I know top-k is a recall-precision balance and a reasonable starting range.
- [ ] I can explain why hybrid search recovers exact matches.
- [ ] I can apply the two-step logic of re-ranking (broad → smart narrow).
- [ ] I recognize the need to rewrite follow-up questions without context.
- [ ] I confirm that I added heavy techniques by measuring, not by measuring.