Position: Make Planning Research Rigorous Again!

In over sixty years since its inception, the field of planning has made significant contributions to both the theory and practice of building planning software that can solve a never-before-seen planning problem. This was done through established practices of rigorous design and evaluation of planning systems. It is therefore our position that this rigor should be employed to the current trend of work on planning with language models. One suggested way to do so is by correctly incorporated the insights, tools, and data from the automated planning community into the design and evaluation of LLM-based planners. We hold that the work from the planning community is not just important from a historical perspective, but includes lessons that could help accelerate the development of planners based on LLMs. This position is particularly important given the fact that we see many of the recent works in the area replicate and propagate mistakes that have been identified within the planning community. By avoiding such known pitfalls, we believe that the works in this space will not only make progress in building LLM-based planners, but planning in general.