Recall Is Not Enough: Token-Centric Metrics for Agentic Schema

LLM-based agents increasingly mediate access to enterprise data through iterative reasoning–acting loops, shifting the primary bottleneck from disk I/O to metadata volume, prefill latency, and token costs—a friction we term the Agent–Data Mismatch. We make two contributions. First, we formalize token-centric evaluation metrics—recall-per-token, cumulative token burden, and cost-per-reasoning-step—that expose inefficiencies invisible under traditional recall-based evaluation. Second, to demonstrate that these metrics can guide practical design, we present a prototype token-aware semantic pruning layer combining bi-encoder retrieval, cross-attention scoring, and schema graph closure. On 1,000 BIRD-SQL queries, the prototype reduces metadata by 3.3–8.3× while maintaining 85–91% schema recall at ≈100 ms overhead, achieving a 2.4–7× recall-per-token advantage over existing methods. Downstream SQL evaluation confirms that the methods ranked highest by our token-centric metrics also deliver the best cost-adjusted execution accuracy.