Workshop paper

Can Skill-Driven LLM Agents Solve Text2SPARQL through Autonomous Exploration and Iterative Refinement?

Abstract

Knowledge Base Question Answering (KBQA) requires systems to answer natural-language questions on struc- tured knowledge graphs such as Wikidata. In Text2SPARQL settings, this task requires generating an executable SPARQL query that can extract the intended answer. However, formulating the correct SPARQL query often requires multiple steps of exploration and reasoning: linking elements of the natural-language question to the appropriate entities and properties, determining how the relevant facts are modeled in Wikidata, identifying a feasible path to the answer among alternative graph patterns, and interpreting intermediate results to refine the query-generation strategy. This paper presents an agentic, skill-driven approach to the WikiKGQA Text2SPARQL task. Rather than treating query generation as a single-step translation problem, the proposed system decomposes the process into a sequence of capabilities, including entity and relation search, graph exploration, query construction, endpoint execution, result validation, and iterative refinement. We evaluate the approach on the English WikiKGQA tracks with mention and no-mention annotations (i.e., pre-supplied entity and property links, including superlatives and quantifiers) and analyze the resulting agent trajectories to determine when autonomous exploration contributes to successful query generation. The proposed approach achieves an F1 score of 0.88 in the with-mentions setting, ranking first in the WikiKGQA: Wiki-Based Knowledge Graph Question Answering Challenge, and an F1 score of 0.81 in the no- mentions setting, ranking second. These results are obtained using a comparatively simple skill-based agentic configuration, without model fine-tuning or complex task-specific pipelines. Our case studies further show that explicit skills and execution-grounded feedback enable a frontier language model to discover Wikidata-specific modeling patterns, revise incorrect assumptions based on graph observations and query results, and generate accurate SPARQL queries for complex KBQA questions.