Neurosymbolic Reasoning with Incremental Knowledge for Sample Efficient Hierarchical Reinforcement Learning

Subrat Prasad Panda1	Blaise Genest2	Arvind Easwaran1

1 NTU, Singapore	2 CNRS, IPAL, France and CNRS@CREATE, Singapore

ECML-PKDD 2026

---

Abstract

(Flat) Reinforcement Learning (RL) agents face significant challenges in environments with sparse rewards that require long-horizon reasoning. A compelling approach to improve sample efficiency is to incorporate knowledge into learning and decision-making. In standard Hierarchical RL (HRL), knowledge is encoded in a fixed, non-updatable form, such as architectural choices, and remains unchanged throughout learning. With fixed HRL, reasoning with incremental knowledge learned during exploration is impractical before sufficient environmental knowledge is acquired, leading to poor sample efficiency. In this work, we propose neurosymbolic HRL with Incremental Knowledge (InK): symbolic high-level components perform symbolic planning (e.g. using D*) on an updatable representation of current InK, while low-level goal-conditioned neural modules learn motion primitives through experience using reward shaping. Experiments on navigation tasks demonstrate that incorporating InK substantially improves sample efficiency. Additionally, to perform optimal symbolic planning given prior knowledge about the world, we develop Belief World Tree Search.

---

Paper, code and citation

Subrat Prasad Panda, Blaise Genest and Arvind Easwaran Neurosymbolic Reasoning with Incremental Knowledge for Sample Efficient Hierarchical Reinforcement Learning European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD), 2026 [PDF] [arXiv] [Code] [Bibtex] [Poster]

---