Ledger RAG
A retrieval pipeline that streams citations into the answer, so every claim is traceable.
- Domain
- AI / LLM
- Status
- published
Python · pgvector · FastAPI
Built after one too many demos where the model was confidently wrong and nobody could tell which document it came from.
Problem
RAG answers were unverifiable: no stable link between a sentence in the answer and a passage in the corpus.
Solution
Attach span-level provenance to every retrieved chunk and stream citations alongside tokens.
Architecture
Hybrid dense + BM25 retrieval, reciprocal rank fusion, cross-encoder rerank, provenance-carrying context assembly.
Key decisions
Kept the reranker small and local rather than calling a hosted API — 40ms of latency was worth the determinism.
Challenges
Chunk boundaries. Structure-aware splitting beat token splitting on every metric but required a parser per format.
Results
recall@8 improved from 0.61 to 0.88 on the internal golden set; hallucinated claims dropped sharply.