The Axe Forgets What The Tree Remembers