GRACE-RAG: Governed Retrieval Architecture for Canonical Evidence Synthesis, Enabling Lightweight Deployment in Closed-Domain Institutional Settings
Asit Desai, Prashant Devadiga, Aman Kumar
• arXiv (cs.IR)
GRACE-RAG introduces a retrieval-governed, graph-augmented architecture that resolves structural entity ambiguity offline prior to LLM generation. By restricting the generative model purely to evidence synthesis over dual embedding surfaces, it boosts response quality by up to 20% on mid-scale models. This framework enables self-hosted 24B parameter models to match the structural quality of proprietary frontier systems in complex institutional question answering.
GRACE-RAG: Governed Retrieval Architecture for Canonical Evidence Synthesis, Enabling Lightweight Deployment in Closed-Domain Institutional Settings
Authors: Asit Desai, Prashant Devadiga, Aman Kumar
Journal: arXiv (cs.IR)
Published: May 2026
Decoupling Structural Reasoning from Generation
In entity-dense institutional environments, traditional vector-only Retrieval-Augmented Generation (RAG) produces fragmented evidence, forcing large language models to perform implicit structural reasoning at inference time. GRACE-RAG solves this bottleneck by externalizing entity normalization, relationship typing, and boundary alignment into an offline Structure Manufacturing pipeline.
Unlike conventional hybrid designs that concatenate raw graph traversals with vector chunks, GRACE-RAG establishes a strict retrieval-governed boundary where the generative model is restricted purely to evidence synthesis. This ensures that institutional terminology fidelity is encoded within retrieval artifacts rather than delegated to generative model capacity.
Core Architectural Pillars
LLM-Assisted Semantic Chunking: Replaces fixed-window tokenization with context-preserving segments aligned to business rules, definitions, and eligibility constraints.
Dual Retrieval Surfaces: Constructs two independent vector indices one over document chunks and another over single-sentence relationship summaries.
Entity Canonicalization & Topological Densification: Groups surface-form variations (e.g., "UPI PIN", "4-digit PIN") into unified canonical nodes. This process increased the average node degree from 2.1 to 6.0 within an 8,000-entity extraction space, significantly eliminating node fragmentation.
Bounded Hybrid Retrieval: Performs parallel vector search and graph-guided expansion per subquery, unifying candidate contexts through global cross-encoder reranking before generation.
Key Benchmark Results & Metrics
Across evaluations on institutional corpora using Mistral 24B, GPT OSS 120B (MoE), and Gemini 2.5 Flash:
Overall Quality Boost: Yields up to a 20.0% overall performance increase on mid-scale models (Mistral 24B jumped from a baseline score of 6.99 to 8.39).
Structural Superiority: Completeness improved by +25.9%, Depth by +35.4%, and Anticipatory Coverage saw a massive +52.1% gain.
Model Capacity Independence: GRACE-RAG with a self-hosted Mistral 24B (8.39) approaches the performance of baseline Gemini 2.5 Flash (8.40), proving that retrieval architecture governs structural quality over parameter scale.
One-Line Takeaway: GRACE-RAG proves that disciplined offline retrieval structuring allows self-hosted 24B models to match proprietary frontier models on complex institutional QA, substantially reducing computational footprint.