You'd be surprised by how flexible the cannibals are if you leave the door open for them. For instance, if you don't cut down the trees that are close to your walls, cannibals can climb the tree and jump into your base. And when a mutant attacks your wall, it will pummel any structure directly on the other side.