Built a LangGraph courtroom over a real legal corpus, where a moderator node acts as orchestra director and schedules exactly the number of perspective nodes requested. Replaced prompt-based constraints with a strict state schema tracking perspective count, jury type, and case category as enforced state rather than a request.
Case Study / LangGraph system
RabbitHole
Contested legal questions resist single consensus answers, and multi-agent systems steered by prompt instructions alone drift. Asked for 2 perspectives, the graph generated 6 to 8, a 3 to 4x overrun that exhausted Groq's token-per-minute ceiling before the debate resolved.
Hierarchical LangGraph orchestration over hybrid retrieval (Pinecone dense vectors plus a BM25 sparse encoder), Jina cross-relevance reranking, a CRAG fallback to Jina Web Search when the grader rejects local chunks, and dual-tier model routing that sends only core synthesis to Llama-3.3-70B.
Cut mean time to verdict from 19.8s to 9.8s, roughly 51%, and eliminated the perspective overrun entirely. Ambiguous retrieval runs the local and web paths in parallel instead of failing.
Technical highlights
What to inspect.
01
Strict state schema replacing prompt-based perspective constraints, which is what actually stopped the 3 to 4x token overrun.
02
Hybrid retrieval pairing dense embeddings for semantic match with BM25 sparse terms for exact legal section and case citations.
03
Three-way CRAG routing: good chunks synthesize locally, bad chunks fall back to web search, ambiguous chunks run both paths in parallel.
04
Dual-tier routing keeping the 70B model on core synthesis only, designed against Groq's 30 req/min and 6,000 token/min free-tier limits.