agentsop-hybrid-retrieval
Enhancement-overlay SOP for adding sparse (BM25 / keyword) retrieval alongside dense (embedding) retrieval. Activate when a calling agent is building, reviewing, or debugging a retrieval pipeline whose corpus contains exact-match tokens — identifiers, error codes, SKUs, API/function names, proper nouns, citations, rare jargon — that pure dense embedding silently misses. Encodes the single decision rule (**hybrid is traffic-driven, not theoretical: add sparse only when the query share that depends on exact tokens is non-trivial**), the wiring of QueryFusionRetriever-style fusion (RRF vs alpha-weighted), and per-query-type alpha tuning. Frame the work as recovering lexical identity that dense pooling destroys, not as "add keyword search for completeness". Cross-links [[llamaindex]].
agentsope · 已有 AI 审核记录
安装与使用
运行命令前先阅读来源与依赖说明。复制命令不会自动安装或执行。
npx skills add agentsope/SkillAlchemy --skill agentsop-hybrid-retrieval技能说明
name: agentsop-hybrid-retrieval version: 0.1.0 description: | Enhancement-overlay SOP for adding sparse (BM25 / keyword) retrieval alongside dense (embedding) retrieval. Activate when a calling agent is building, reviewing, or debugging a retrieval pipeline whose corpus contains exact-match tokens — identifiers, error codes, SKUs, API/function names, proper nouns, citations, rare jargon — that pure dense embedding silently misses. Encodes the single decision rule (hybrid is traffic-driven, not theoretical: add sparse only when the query share that depends on exact tokens is non-trivial), the wiring of QueryFusionRetriever-style fusion (RRF vs alpha-weighted), and per-query-type alpha tuning. Frame the work as recovering lexical identity that dense pooling destroys, not as "add keyword search for completeness". Cross-links [[llamaindex]]. overlay: true cross_links: [llamaindex, langchain] trigger_keywords:
- "hybrid search"
- "hybrid retrieval"
- "BM25"
- "sparse retrieval"
- "dense plus sparse"
- "QueryFusionRetriever"
- "EnsembleRetriever"
- "RRF"
- "reciprocal rank fusion"
- "alpha tuning"
- "keyword search RAG" when_to_use:
- "the corpus contains exact-match tokens: identifiers, error codes, SKUs, part numbers, API/function names, proper nouns, legal/medical citations, rare jargon"
- "users report 'I searched the exact code/name and got nothing / the wrong doc' on a dense-only retriever"
- "reviewing a retriever where traffic includes lexical-identity lookups but only embeddings are wired"
- "tuning recall on a corpus where both meaning AND exact strings matter"
- "deciding between pure dense, hybrid, or sparse-first for a new RAG corpus" when_not_to_use:
- "traffic is purely semantic / conceptual with <5% lexical-identity queries — hybrid is over-engineering"
- "the corpus has no stable identifiers and queries never reference exact strings"
- "pre-baseline: ship dense-only and measure first; hybrid is a Stage-3 optimization (see [[llamaindex]])"
Hybrid Retrieval · Dense + Sparse SOP
Third-person operating model for a coder agent that owns retrieval recall on a corpus where both meaning and exact tokens matter. The audience is the LLM agent writing or reviewing retrieval code — not an end user.
One sentence: Dense captures meaning, sparse captures exact tokens; hybrid wins when both matter — but only fuse them when traffic actually carries exact-match queries, and tune the blend per query type or hybrid loses to dense.
1. 何时激活 (Activation Rules)
Activate this skill when any of the following holds:
- The corpus contains exact-match tokens that a query may reference verbatim:
error codes (
ERR_SSL_PROTOCOL), SKUs / part numbers (A1-2293-X), API or function names (as_query_engine), proper nouns, legal/medical citations (42 U.S.C. § 1983), version strings, rare jargon, ticket IDs. - A bug report says "I searched the exact code/name/string and got nothing", or "the right document exists but dense retrieval ranks it below fuzzy near-misses".
- PR review surfaces a retriever serving lexical-identity traffic but wired
dense-only (
index.as_retriever(...)/similarity_search(...)with no sparse leg). - You are tuning recall and have already exhausted the cheap dense knobs (prompt, embedding model, chunk size) per the [[llamaindex]] optimization ladder — hybrid is the next rung.
- The user mentions hybrid search, BM25, sparse retrieval, RRF,
QueryFusionRetriever,EnsembleRetriever, or vector-store-native hybrid (Qdrant/Weaviate/Pinecone).
Do not activate when:
- Traffic is purely semantic ("what does X mean?", "summarize the policy") with a lexical-identity share <5% — adding BM25 doubles index footprint for no gain.
- The corpus has no stable identifiers and no query ever quotes an exact string.
- No dense baseline + eval loop exists yet. Hybrid is a Stage-3 optimization ([[llamaindex]] Stage 3 step 4): baseline and measure before fusing.
2. 核心心智模型 (Core Mental Model)
Three principles. Violating any of them is why teams "try hybrid and conclude it didn't help".
Principle 1 — Dense and sparse fail in opposite directions
Dense embedding models destroy lexical identity by pooling token representations: querying a specific error string yields a vector that captures "document about SSL errors" rather than "document containing this exact string". BM25 does the inverse — it scores against an inverted index of exact tokens and is blind to synonyms and paraphrase. (TianPan, Hybrid search in production, 2026; cited in [[llamaindex]] Dilemma 2.)
Operational corollary: the symptom "I pasted the exact code and got nothing" is not a bug in the embedding model — it is the embedding model working as designed. The fix is a second retriever that indexes tokens, not a better embedding.
Principle 2 — Hybrid is traffic-driven, not theoretical
Whether to add sparse is decided by the query-type distribution of real traffic, not by a belief that "more retrievers = better". The decision threshold is the lexical share: the fraction of queries whose correct answer hinges on an exact token. Below ~5% → dense-only. 5–50% → hybrid. Above ~50% (code, logs, legal) → invert to sparse-first with dense as a rerank signal. (Cited in [[llamaindex]] OP-04 / Dilemma 2.)
Principle 3 — One global alpha underperforms; tune per query type
alpha is the dense↔sparse blend (alpha=1 → pure dense, alpha=0 → pure sparse).
A semantic query wants high alpha; a lexical-identity query wants low alpha. A single
global alpha picked to help lexical queries hurts the semantic slice — which is
exactly why a flat alpha "loses to pure dense" and teams wrongly conclude hybrid
failed. Tune alpha per query type, or route per type and pick alpha per route.
(LlamaIndex alpha-tuning blog; [[llamain