Ledger RAG

A retrieval pipeline that streams citations into the answer, so every claim is traceable.

Domain
AI / LLM
Status
published

Python · pgvector · FastAPI

Built after one too many demos where the model was confidently wrong and nobody could tell which document it came from.

Problem

RAG answers were unverifiable: no stable link between a sentence in the answer and a passage in the corpus.

Solution

Attach span-level provenance to every retrieved chunk and stream citations alongside tokens.

Architecture

Hybrid dense + BM25 retrieval, reciprocal rank fusion, cross-encoder rerank, provenance-carrying context assembly.

Key decisions

Kept the reranker small and local rather than calling a hosted API — 40ms of latency was worth the determinism.

Challenges

Chunk boundaries. Structure-aware splitting beat token splitting on every metric but required a parser per format.

Results

recall@8 improved from 0.61 to 0.88 on the internal golden set; hallucinated claims dropped sharply.

ShareXLinkedIn