I think a father-child relationship isn't off the books for this story, but it sure as hell shouldn't happen immediately. I vaguely get the emperor's decision to ignore the twins so that he doesn't get emotionally attached and therefore gains a weakness that can be exploited but like. Dude. Don't be so fucking wishy washy about it. Either treat them like your kids or ignore them entirely, none of this "abandon them to their shitty mom for five years and then decide to raise them in the palace" garbage
tbh the dude should've just killed Rurahel and raised the kids in the palace, if he a) didn't want Rurahel to worm her way into the palace and b) didn't want the kids to die or get mistreated. He's not unsalvageable as a dad, but he is a neglectful, incompetent dumbass