Concept guide · Embeddings and vector search
What is dense retrieval versus sparse retrieval?
A slower explanation for when the short interview answer is not enough. Start with the intuition, then connect it to the technical idea and the way an interviewer expects you to reason about it.
ML area
Generative AI → Dense and sparse retrieval
Interview format
Core · Concept
Question
GENAI-036
Start with the idea
In simple terms
Dense retrieval searches learned vectors for similar meaning. Sparse retrieval searches words and terms, which is often better for exact names, error codes, IDs, and rare phrases.
Make it concrete
A dense retriever may understand “reset my password,” while a sparse retriever is more likely to preserve an exact error code such as E403-17.
Build the answer
How to reason about it
- 1Identify whether the query depends on meaning, exact tokens, or both.
- 2Choose dense, sparse, or hybrid retrieval based on the failure pattern.
- 3Evaluate recall separately for semantic paraphrases and exact-match cases.
Go one level deeper
The technical explanation
Dense retrieval compresses semantic relationships into continuous vectors, but that compression can blur exact constraints. Sparse lexical methods reward term overlap and can handle identifiers that have little semantic neighbourhood. A production RAG system often combines both candidate sets, because user questions mix intent, names, numbers, and domain vocabulary.
Say this in the interview
Dense retrieval compares learned vectors and handles semantic similarity. Sparse retrieval, such as term-based scoring, is strong for exact words, identifiers, names, and rare terms. Dense search can miss exact constraints, while sparse search can miss paraphrases, so real systems often combine them.
Keep preparing