There's no clear reason why Zed left, but he does mention during one of Olivia's flashback that beings like him eat something akin to souls after a person dies. However, there is nuance to the flavor of the souls, where the stronger or more brilliant the soul the more delicious it is. So Olivia indirectly seeks stronger opponents to kill so that Zed can have a wonderful meal.
But for more plot related reasons, one of the Dukes in the evil Empire has connections to one of Zed's companions who often questioned his queer habits of "observing" things. This other being, is using the Duke and his influence within the empire to stir up wars to create a massive amount of casualties. So I guess it's something akin to "Quality vs Quantity" in their methodology of harvesting souls?