I slightly doubt he would leave those girls but if he allows those spirit users to be enslaved, I'm dropping this one. Letting someone else's life get ruined to protect your own when you could have done something about it is the wrong kind of character development.
........Seriously who am I...